Skip to content

Commit e953fa5

Browse files
donislawdevclaude
andauthored
format: a JSON document can be minified or indented (#84)
* format: a JSON document can be minified or indented json gains a formatting setting taking record-per-line, minified or indented, with the layout literals held on a style so the arithmetic that hits an exact byte count measures whichever one is in use rather than predicting it. The default is the layout this format has always written, pinned by two new golden values. The measurement that shaped this is a negative one: no reader can tell the three apart. The same records minified, one per line, indented by two and indented by four all parse in CPython's json and in V8. So the value of the setting is entirely outside the parser - a minified document of any size is one line and ends without a newline, an indented one holds about a third fewer records in the same bytes - and the structural checker has to be TOLD which layout to expect, for the reason the CSV dialect is told. That immediately falsified what the existing checker said about itself. It read "the manifest states one record per line" and counted lines against it, so both new layouts would have broken it. The sentence was not wrong - it stopped being true the day the layout became something a person can ask for. Each layout answers for its own floor: 216 B minified, 219 B a record per line, 318 B indented. The registry declares the default layout's, the generator answers for the rest, and the refusal names the layout it is about because the same size is legal in another. The canary is the half that makes the rest mean anything: a checker handed a file and the RIGHT layout name would pass even if it ignored the name, so the guard hands it every wrong one instead. Six pairs, six refusals. Six mutations, all caught - including one already in the list that this refactor had staled, which staleness.py found rather than a reader. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: say out loud that the window offers the JSON layout setting The engine gained property:json.formatting and neither parity list named it, so TestEveryEngineCapabilityIsClassifiedForBothSurfaces refused the run on all four jobs. That is the guard doing its job: a setting reaching the engine without anybody answering whether the window offers it is exactly the drift D1 forbids. The bar for the reachable list is two things rather than one - a control on the screen, and a guard that presses it and finds the value on the other side - and both were measured before the name went in. The field is drawn from the declaration and nothing else, the menu opens on its declared default and is named among the twenty that do, and the path from a field to the manifest is pinned by the guard that types a value and reads it back off the disk. One thing was worth asking rather than assuming. The window sends every setting, because a menu cannot be empty, so a run started there says record-per-line out loud where the command line says nothing at all. Those two have to be one file, and the guard that pins them already exists. D1 parity: 114 of 125 capabilities reachable from the window, eleven still to go. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent be665f5 commit e953fa5

11 files changed

Lines changed: 538 additions & 73 deletions

File tree

CHANGELOG.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,21 @@ because it turns other people's test suites red.
8383
generated text is English, so a file written in one would be byte for byte
8484
the same file as UTF-8 - a setting that changes nothing.
8585

86+
- **JSON documents can be minified or indented.** A `formatting` setting on
87+
`json` taking `record-per-line`, `minified` or `indented`. It defaults to the
88+
one record per line this format has always written, so a recipe that says
89+
nothing gets the same bytes.
90+
91+
```
92+
tfg generate --format json --size 1mb --set formatting=minified
93+
```
94+
95+
Every reader accepts all three, which is the point: what changes is
96+
everything around the parser. A minified document of any size is one single
97+
line and ends without a newline, and an indented one holds roughly a third
98+
fewer records in the same number of bytes. The smallest document each layout
99+
can produce differs too, and asking for less names the layout it is about.
100+
86101
### Security
87102

88103
- **On Windows, the desktop window loads the library it uses for dark menus from

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -477,7 +477,8 @@ recipe. `tfg formats <id>` prints the allowed range or list for each:
477477
| `csv` | `delimiter`, `line_ending`, `header`, `quote_style`, `columns` |
478478
| `log` | `entry_format`, `timestamps`, `rate`, `methods`, `status_mix`, `level_mix`, `ip_version`, `line_ending` |
479479
| `txt`, `md` | `encoding`, `bom` |
480-
| `json`, `xml`, `html`, `svg` | none |
480+
| `json` | `formatting` |
481+
| `xml`, `html`, `svg` | none |
481482
482483
```
483484
tfg generate --format jpg --size 500kb --set width=1920 --set height=1080 --set quality=85

internal/format/jsonfile/json.go

Lines changed: 59 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -38,10 +38,6 @@ import (
3838
const (
3939
generatorVersion = "1"
4040

41-
// The document is an array with one record per line. Minified on one line
42-
// and indented forms come later as a property.
43-
prologue = "[\n"
44-
4541
emailDomain = "@example.com"
4642

4743
// Fixed widths, so the parts that are not the note stay predictable.
@@ -54,36 +50,14 @@ const (
5450
// be asked for.
5551
maxIDDigits = 19
5652

57-
// The literal parts of a record, named so the arithmetic below is a
58-
// constant expression rather than a number somebody has to keep in step.
59-
openID = `{"id":`
60-
openName = `,"name":"`
61-
openMail = `","email":"`
62-
openAmt = `","amount":`
63-
openAct = `,"active":`
64-
nullPart = `,"retired":null`
65-
openTags = `,"tags":["`
66-
tagSep = `","`
67-
openAddr = `"],"address":{"city":"`
68-
openZip = `","zip":"`
69-
openNote = `"},"note":"`
70-
closeRec = `"}`
71-
72-
// A record either has another one after it or closes the array.
73-
tailMore = closeRec + ",\n"
74-
tailLast = closeRec + "\n]\n"
75-
7653
// widestBool is "false", the longer of the two.
7754
widestBool = 5
78-
79-
// fixedWidth is every literal byte of a closing record - everything except
80-
// the record number, the name, the two tags, the city and the note.
81-
fixedWidth = len(openID) + len(openName) + len(openMail) + len(emailDomain) +
82-
len(openAmt) + amountWidth + len(openAct) + widestBool + len(nullPart) +
83-
len(openTags) + len(tagSep) + len(openAddr) + len(openZip) + zipWidth +
84-
len(openNote) + len(tailLast)
8555
)
8656

57+
// The literal parts of a record live on the style, because there are three
58+
// layouts of them and the arithmetic has to measure whichever one is in use.
59+
// See style.go.
60+
8761
func init() {
8862
format.Register(format.Descriptor{
8963
ID: "json",
@@ -95,7 +69,12 @@ func init() {
9569
// by naming a byte count - that is a shape request, and it arrives with
9670
// the record count property. The minimum here is the array and one whole
9771
// record.
98-
MinBytes: minimumBytes(),
72+
//
73+
// The DEFAULT layout's minimum, the way CSV declares its default
74+
// dialect's. Every other layout answers for itself, through the same
75+
// refusal, and SmallestAccepted asks the generator rather than reading
76+
// this number.
77+
MinBytes: minimumBytes(defaultStyle()),
9978

10079
Padding: format.PaddingChannel{
10180
Name: "the note value of the last record",
@@ -107,27 +86,38 @@ func init() {
10786
// structure under test. The file name and the manifest carry it instead.
10887
Label: format.LabelExternalOnly,
10988
Oracle: "node-json",
110-
// Nesting depth, key counts, value types, indentation and NDJSON come
111-
// later. Declaring none now makes a recipe asking for them fail loudly.
112-
Properties: nil,
89+
// Nesting depth, key counts, value types and NDJSON come later.
90+
// Declaring only what is here makes a recipe asking for them fail
91+
// loudly rather than quietly producing something else.
92+
Properties: properties(),
11393
GeneratorVersion: generatorVersion,
11494
Generator: generator{},
11595
})
11696
}
11797

11898
type generator struct{}
11999

120-
type memo struct{ seed uint64 }
100+
type memo struct {
101+
seed uint64
102+
s style
103+
}
121104

122105
func (generator) Plan(r format.Request) (format.Plan, error) {
123-
min := minimumBytes()
106+
s, err := parseStyle(r.Properties)
107+
if err != nil {
108+
return format.Plan{}, err
109+
}
110+
111+
min := minimumBytes(s)
124112
if r.Bytes < min {
125113
return format.Plan{}, &format.BelowMinimumError{
126114
Format: "JSON",
127115
Requested: r.Bytes,
128116
Minimum: min,
129-
Reason: "a document holds whole records and one record with every value type needs that much",
130-
Hint: fmt.Sprintf("Ask for %d B or more.", min),
117+
Reason: fmt.Sprintf(
118+
"a document holds whole records, and one %s record with every value type needs that much",
119+
s.name),
120+
Hint: fmt.Sprintf("Ask for %d B or more.", min),
131121
}
132122
}
133123

@@ -136,15 +126,15 @@ func (generator) Plan(r format.Request) (format.Plan, error) {
136126
Exact: true,
137127
Determinism: format.DeterminismByte,
138128
Properties: map[string]any{
139-
"encoding": "utf-8",
140-
"formatting": "record-per-line",
141-
"root": "array",
142-
"depth": 3,
129+
"encoding": "utf-8",
130+
Formatting: s.name,
131+
"root": "array",
132+
"depth": 3,
143133
// Stated even though it is always false here, so a test can assert
144134
// on it without knowing which formats carry a label internally.
145135
format.PropertyLabelEmbedded: false,
146136
},
147-
Memo: memo{seed: r.Seed},
137+
Memo: memo{seed: r.Seed, s: s},
148138
}, nil
149139
}
150140

@@ -154,24 +144,28 @@ func (generator) Write(ctx context.Context, w io.Writer, p format.Plan) error {
154144
return fmt.Errorf("json: the plan was not produced by this generator")
155145
}
156146

157-
if err := core.WriteAll(w, []byte(prologue)); err != nil {
147+
if err := core.WriteAll(w, []byte(m.s.prologue)); err != nil {
158148
return err
159149
}
160150

161151
rng := core.NewRand(m.seed)
162-
return core.FillRecords(ctx, w, rng, p.Bytes-int64(len(prologue)), &records{})
152+
return core.FillRecords(ctx, w, rng, p.Bytes-int64(len(m.s.prologue)), &records{s: m.s})
163153
}
164154

165155
// records builds the objects inside the array. It carries the record number, so
166-
// the id counts up the way a real export does.
167-
type records struct{ next int64 }
156+
// the id counts up the way a real export does, and the layout the document is
157+
// being written in.
158+
type records struct {
159+
next int64
160+
s style
161+
}
168162

169163
// Shortest is the smallest record this builder can close a document with: the
170164
// widest record number, the longest word in all five places a word appears, the
171165
// longer of the two booleans, and an empty note. It has to hold for every draw
172166
// rather than for the lucky one.
173167
func (r *records) Shortest() int64 {
174-
return int64(maxIDDigits + 5*longestWord + fixedWidth)
168+
return int64(maxIDDigits + 5*longestWord + r.s.fixed())
175169
}
176170

177171
func (r *records) Append(dst []byte, rng *rand.Rand) []byte {
@@ -211,47 +205,47 @@ func (r *records) append(dst []byte, rng *rand.Rand, want int64) []byte {
211205
city := words[rng.IntN(len(words))]
212206
zip := 10000 + rng.IntN(90000)
213207

214-
dst = append(dst, openID...)
208+
dst = append(dst, r.s.openID...)
215209
dst = strconv.AppendInt(dst, r.next, 10)
216-
dst = append(dst, openName...)
210+
dst = append(dst, r.s.openName...)
217211
dst = append(dst, name...)
218-
dst = append(dst, openMail...)
212+
dst = append(dst, r.s.openMail...)
219213
dst = append(dst, name...)
220214
dst = append(dst, emailDomain...)
221-
dst = append(dst, openAmt...)
215+
dst = append(dst, r.s.openAmt...)
222216
dst = strconv.AppendInt(dst, int64(whole), 10)
223217
dst = append(dst, '.')
224218
if cents < 10 {
225219
dst = append(dst, '0')
226220
}
227221
dst = strconv.AppendInt(dst, int64(cents), 10)
228-
dst = append(dst, openAct...)
222+
dst = append(dst, r.s.openAct...)
229223
if active {
230224
dst = append(dst, "true"...)
231225
} else {
232226
dst = append(dst, "false"...)
233227
}
234-
dst = append(dst, nullPart...)
235-
dst = append(dst, openTags...)
228+
dst = append(dst, r.s.nullPart...)
229+
dst = append(dst, r.s.openTags...)
236230
dst = append(dst, tagA...)
237-
dst = append(dst, tagSep...)
231+
dst = append(dst, r.s.tagSep...)
238232
dst = append(dst, tagB...)
239-
dst = append(dst, openAddr...)
233+
dst = append(dst, r.s.openAddr...)
240234
dst = append(dst, city...)
241-
dst = append(dst, openZip...)
235+
dst = append(dst, r.s.openZip...)
242236
dst = strconv.AppendInt(dst, int64(zip), 10)
243-
dst = append(dst, openNote...)
237+
dst = append(dst, r.s.openNote...)
244238

245239
if want < 0 {
246240
dst = appendPhrase(dst, rng, 3+rng.IntN(5))
247-
return append(dst, tailMore...)
241+
return append(dst, r.s.tailMore...)
248242
}
249243

250244
// Everything written so far, plus the bytes that close the record and the
251245
// array.
252-
used := int64(len(dst)-start) + int64(len(tailLast))
246+
used := int64(len(dst)-start) + int64(len(r.s.tailLast))
253247
dst = appendFiller(dst, want-used)
254-
return append(dst, tailLast...)
248+
return append(dst, r.s.tailLast...)
255249
}
256250

257251
// appendPhrase writes a readable note. Words and single spaces only - a JSON
@@ -281,9 +275,9 @@ func appendFiller(dst []byte, n int64) []byte {
281275
// minimumBytes is the opening bracket and one whole record, computed rather
282276
// than written down so it cannot drift away from the template the way a number
283277
// in a document would.
284-
func minimumBytes() int64 {
285-
var r records
286-
return int64(len(prologue)) + r.Shortest()
278+
func minimumBytes(s style) int64 {
279+
r := records{s: s}
280+
return int64(len(s.prologue)) + r.Shortest()
287281
}
288282

289283
// longestWord is the widest draw, because the minimum has to hold for every

0 commit comments

Comments
 (0)