From d50eb0b155efd5b438097bceb70aeb94ec850de6 Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Mon, 7 Sep 2026 20:47:53 +0200 Subject: [PATCH 1/2] format: a JSON document can be minified or indented json gains a formatting setting taking record-per-line, minified or indented, with the layout literals held on a style so the arithmetic that hits an exact byte count measures whichever one is in use rather than predicting it. The default is the layout this format has always written, pinned by two new golden values. The measurement that shaped this is a negative one: no reader can tell the three apart. The same records minified, one per line, indented by two and indented by four all parse in CPython's json and in V8. So the value of the setting is entirely outside the parser - a minified document of any size is one line and ends without a newline, an indented one holds about a third fewer records in the same bytes - and the structural checker has to be TOLD which layout to expect, for the reason the CSV dialect is told. That immediately falsified what the existing checker said about itself. It read "the manifest states one record per line" and counted lines against it, so both new layouts would have broken it. The sentence was not wrong - it stopped being true the day the layout became something a person can ask for. Each layout answers for its own floor: 216 B minified, 219 B a record per line, 318 B indented. The registry declares the default layout's, the generator answers for the rest, and the refusal names the layout it is about because the same size is legal in another. The canary is the half that makes the rest mean anything: a checker handed a file and the RIGHT layout name would pass even if it ignored the name, so the guard hands it every wrong one instead. Six pairs, six refusals. Six mutations, all caught - including one already in the list that this refactor had staled, which staleness.py found rather than a reader. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 15 ++ README.md | 3 +- internal/format/jsonfile/json.go | 124 +++++----- internal/format/jsonfile/style.go | 180 ++++++++++++++ internal/guard/generatorbytes_test.go | 7 + internal/guard/jsonlayout_test.go | 224 ++++++++++++++++++ internal/guard/testdata/generator-golden.json | 8 + internal/oracle/strict.py | 39 ++- web/public/formats/index.html | 5 + web/public/pl/formaty/index.html | 5 + 10 files changed, 537 insertions(+), 73 deletions(-) create mode 100644 internal/format/jsonfile/style.go create mode 100644 internal/guard/jsonlayout_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index c598c44..5844e36 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,6 +83,21 @@ because it turns other people's test suites red. generated text is English, so a file written in one would be byte for byte the same file as UTF-8 - a setting that changes nothing. +- **JSON documents can be minified or indented.** A `formatting` setting on + `json` taking `record-per-line`, `minified` or `indented`. It defaults to the + one record per line this format has always written, so a recipe that says + nothing gets the same bytes. + + ``` + tfg generate --format json --size 1mb --set formatting=minified + ``` + + Every reader accepts all three, which is the point: what changes is + everything around the parser. A minified document of any size is one single + line and ends without a newline, and an indented one holds roughly a third + fewer records in the same number of bytes. The smallest document each layout + can produce differs too, and asking for less names the layout it is about. + ### Security - **On Windows, the desktop window loads the library it uses for dark menus from diff --git a/README.md b/README.md index 7f89444..7d064c7 100644 --- a/README.md +++ b/README.md @@ -477,7 +477,8 @@ recipe. `tfg formats ` prints the allowed range or list for each: | `csv` | `delimiter`, `line_ending`, `header`, `quote_style`, `columns` | | `log` | `entry_format`, `timestamps`, `rate`, `methods`, `status_mix`, `level_mix`, `ip_version`, `line_ending` | | `txt`, `md` | `encoding`, `bom` | -| `json`, `xml`, `html`, `svg` | none | +| `json` | `formatting` | +| `xml`, `html`, `svg` | none | ``` tfg generate --format jpg --size 500kb --set width=1920 --set height=1080 --set quality=85 diff --git a/internal/format/jsonfile/json.go b/internal/format/jsonfile/json.go index 49e86ec..152578b 100644 --- a/internal/format/jsonfile/json.go +++ b/internal/format/jsonfile/json.go @@ -38,10 +38,6 @@ import ( const ( generatorVersion = "1" - // The document is an array with one record per line. Minified on one line - // and indented forms come later as a property. - prologue = "[\n" - emailDomain = "@example.com" // Fixed widths, so the parts that are not the note stay predictable. @@ -54,36 +50,14 @@ const ( // be asked for. maxIDDigits = 19 - // The literal parts of a record, named so the arithmetic below is a - // constant expression rather than a number somebody has to keep in step. - openID = `{"id":` - openName = `,"name":"` - openMail = `","email":"` - openAmt = `","amount":` - openAct = `,"active":` - nullPart = `,"retired":null` - openTags = `,"tags":["` - tagSep = `","` - openAddr = `"],"address":{"city":"` - openZip = `","zip":"` - openNote = `"},"note":"` - closeRec = `"}` - - // A record either has another one after it or closes the array. - tailMore = closeRec + ",\n" - tailLast = closeRec + "\n]\n" - // widestBool is "false", the longer of the two. widestBool = 5 - - // fixedWidth is every literal byte of a closing record - everything except - // the record number, the name, the two tags, the city and the note. - fixedWidth = len(openID) + len(openName) + len(openMail) + len(emailDomain) + - len(openAmt) + amountWidth + len(openAct) + widestBool + len(nullPart) + - len(openTags) + len(tagSep) + len(openAddr) + len(openZip) + zipWidth + - len(openNote) + len(tailLast) ) +// The literal parts of a record live on the style, because there are three +// layouts of them and the arithmetic has to measure whichever one is in use. +// See style.go. + func init() { format.Register(format.Descriptor{ ID: "json", @@ -95,7 +69,12 @@ func init() { // by naming a byte count - that is a shape request, and it arrives with // the record count property. The minimum here is the array and one whole // record. - MinBytes: minimumBytes(), + // + // The DEFAULT layout's minimum, the way CSV declares its default + // dialect's. Every other layout answers for itself, through the same + // refusal, and SmallestAccepted asks the generator rather than reading + // this number. + MinBytes: minimumBytes(defaultStyle()), Padding: format.PaddingChannel{ Name: "the note value of the last record", @@ -107,9 +86,10 @@ func init() { // structure under test. The file name and the manifest carry it instead. Label: format.LabelExternalOnly, Oracle: "node-json", - // Nesting depth, key counts, value types, indentation and NDJSON come - // later. Declaring none now makes a recipe asking for them fail loudly. - Properties: nil, + // Nesting depth, key counts, value types and NDJSON come later. + // Declaring only what is here makes a recipe asking for them fail + // loudly rather than quietly producing something else. + Properties: properties(), GeneratorVersion: generatorVersion, Generator: generator{}, }) @@ -117,17 +97,27 @@ func init() { type generator struct{} -type memo struct{ seed uint64 } +type memo struct { + seed uint64 + s style +} func (generator) Plan(r format.Request) (format.Plan, error) { - min := minimumBytes() + s, err := parseStyle(r.Properties) + if err != nil { + return format.Plan{}, err + } + + min := minimumBytes(s) if r.Bytes < min { return format.Plan{}, &format.BelowMinimumError{ Format: "JSON", Requested: r.Bytes, Minimum: min, - Reason: "a document holds whole records and one record with every value type needs that much", - Hint: fmt.Sprintf("Ask for %d B or more.", min), + Reason: fmt.Sprintf( + "a document holds whole records, and one %s record with every value type needs that much", + s.name), + Hint: fmt.Sprintf("Ask for %d B or more.", min), } } @@ -136,15 +126,15 @@ func (generator) Plan(r format.Request) (format.Plan, error) { Exact: true, Determinism: format.DeterminismByte, Properties: map[string]any{ - "encoding": "utf-8", - "formatting": "record-per-line", - "root": "array", - "depth": 3, + "encoding": "utf-8", + Formatting: s.name, + "root": "array", + "depth": 3, // Stated even though it is always false here, so a test can assert // on it without knowing which formats carry a label internally. format.PropertyLabelEmbedded: false, }, - Memo: memo{seed: r.Seed}, + Memo: memo{seed: r.Seed, s: s}, }, nil } @@ -154,24 +144,28 @@ func (generator) Write(ctx context.Context, w io.Writer, p format.Plan) error { return fmt.Errorf("json: the plan was not produced by this generator") } - if err := core.WriteAll(w, []byte(prologue)); err != nil { + if err := core.WriteAll(w, []byte(m.s.prologue)); err != nil { return err } rng := core.NewRand(m.seed) - return core.FillRecords(ctx, w, rng, p.Bytes-int64(len(prologue)), &records{}) + return core.FillRecords(ctx, w, rng, p.Bytes-int64(len(m.s.prologue)), &records{s: m.s}) } // records builds the objects inside the array. It carries the record number, so -// the id counts up the way a real export does. -type records struct{ next int64 } +// the id counts up the way a real export does, and the layout the document is +// being written in. +type records struct { + next int64 + s style +} // Shortest is the smallest record this builder can close a document with: the // widest record number, the longest word in all five places a word appears, the // longer of the two booleans, and an empty note. It has to hold for every draw // rather than for the lucky one. func (r *records) Shortest() int64 { - return int64(maxIDDigits + 5*longestWord + fixedWidth) + return int64(maxIDDigits + 5*longestWord + r.s.fixed()) } func (r *records) Append(dst []byte, rng *rand.Rand) []byte { @@ -211,47 +205,47 @@ func (r *records) append(dst []byte, rng *rand.Rand, want int64) []byte { city := words[rng.IntN(len(words))] zip := 10000 + rng.IntN(90000) - dst = append(dst, openID...) + dst = append(dst, r.s.openID...) dst = strconv.AppendInt(dst, r.next, 10) - dst = append(dst, openName...) + dst = append(dst, r.s.openName...) dst = append(dst, name...) - dst = append(dst, openMail...) + dst = append(dst, r.s.openMail...) dst = append(dst, name...) dst = append(dst, emailDomain...) - dst = append(dst, openAmt...) + dst = append(dst, r.s.openAmt...) dst = strconv.AppendInt(dst, int64(whole), 10) dst = append(dst, '.') if cents < 10 { dst = append(dst, '0') } dst = strconv.AppendInt(dst, int64(cents), 10) - dst = append(dst, openAct...) + dst = append(dst, r.s.openAct...) if active { dst = append(dst, "true"...) } else { dst = append(dst, "false"...) } - dst = append(dst, nullPart...) - dst = append(dst, openTags...) + dst = append(dst, r.s.nullPart...) + dst = append(dst, r.s.openTags...) dst = append(dst, tagA...) - dst = append(dst, tagSep...) + dst = append(dst, r.s.tagSep...) dst = append(dst, tagB...) - dst = append(dst, openAddr...) + dst = append(dst, r.s.openAddr...) dst = append(dst, city...) - dst = append(dst, openZip...) + dst = append(dst, r.s.openZip...) dst = strconv.AppendInt(dst, int64(zip), 10) - dst = append(dst, openNote...) + dst = append(dst, r.s.openNote...) if want < 0 { dst = appendPhrase(dst, rng, 3+rng.IntN(5)) - return append(dst, tailMore...) + return append(dst, r.s.tailMore...) } // Everything written so far, plus the bytes that close the record and the // array. - used := int64(len(dst)-start) + int64(len(tailLast)) + used := int64(len(dst)-start) + int64(len(r.s.tailLast)) dst = appendFiller(dst, want-used) - return append(dst, tailLast...) + return append(dst, r.s.tailLast...) } // appendPhrase writes a readable note. Words and single spaces only - a JSON @@ -281,9 +275,9 @@ func appendFiller(dst []byte, n int64) []byte { // minimumBytes is the opening bracket and one whole record, computed rather // than written down so it cannot drift away from the template the way a number // in a document would. -func minimumBytes() int64 { - var r records - return int64(len(prologue)) + r.Shortest() +func minimumBytes(s style) int64 { + r := records{s: s} + return int64(len(s.prologue)) + r.Shortest() } // longestWord is the widest draw, because the minimum has to hold for every diff --git a/internal/format/jsonfile/style.go b/internal/format/jsonfile/style.go new file mode 100644 index 0000000..315e963 --- /dev/null +++ b/internal/format/jsonfile/style.go @@ -0,0 +1,180 @@ +// The layout: the one thing about a JSON document that every reader forgives +// and almost nothing downstream of the reader does. +// +// Measured 2026-09-07 on two implementations in two languages - Python's json +// and V8's JSON.parse - with the same records written four ways: minified, one +// record per line, indented by two and indented by four. All four parse +// everywhere. So this setting is not about whether a parser copes. It is about +// the file: the same three records are 130 B minified and 260 B indented, and a +// minified document of any size is one single line. +// +// That is what makes it worth having. A tester whose pipeline reads line by +// line, splits a file into chunks, diffs two exports or loads one into an +// editor meets a different file in each shape, and every one of them is valid +// JSON that no parser will complain about. +package jsonfile + +import "github.com/donislawdev/TestingFilesGenerator/internal/format" + +// Setting name. A public name, so it is spelled once. +const Formatting = "formatting" + +// The three layouts, spelled as the recipe spells them. +const ( + Indented = "indented" + Minified = "minified" + RecordPerLine = "record-per-line" +) + +// style is every literal that stands between one value of a record and the +// next, plus what opens the document and what closes it. +// +// Held as strings rather than as an indent width and a rule for applying it, +// because the arithmetic that hits an exact byte count has to measure these +// rather than predict them. A width plus a rule is two things that can +// disagree - the literals are one. +type style struct { + name string + prologue string + + openID string + openName string + openMail string + openAmt string + openAct string + nullPart string + openTags string + tagSep string + openAddr string + openZip string + openNote string + + // tailMore closes a record that has another after it, tailLast closes the + // record that closes the document. + tailMore string + tailLast string +} + +// fixed is every literal byte of a closing record: everything except the +// record number, the five words and the note. +// +// Computed from the style's own literals rather than written down beside them, +// so a layout added later cannot carry a number that disagrees with its own +// text. The same reason minimumBytes is computed rather than declared. +func (s style) fixed() int { + return len(s.openID) + len(s.openName) + len(s.openMail) + len(emailDomain) + + len(s.openAmt) + amountWidth + len(s.openAct) + widestBool + len(s.nullPart) + + len(s.openTags) + len(s.tagSep) + len(s.openAddr) + len(s.openZip) + zipWidth + + len(s.openNote) + len(s.tailLast) +} + +// recordPerLine is the layout this format has always written. Leaving the +// setting alone produces the same bytes it always did. +var recordPerLine = style{ + name: RecordPerLine, + prologue: "[\n", + openID: `{"id":`, + openName: `,"name":"`, + openMail: `","email":"`, + openAmt: `","amount":`, + openAct: `,"active":`, + nullPart: `,"retired":null`, + openTags: `,"tags":["`, + tagSep: `","`, + openAddr: `"],"address":{"city":"`, + openZip: `","zip":"`, + openNote: `"},"note":"`, + tailMore: "\"},\n", + tailLast: "\"}\n]\n", +} + +// minified is the same tokens with no whitespace anywhere, which puts the +// whole document on one line. +// +// It ends without a trailing newline, on purpose: a minified document that +// ends in a newline is not minified, and a file with no final newline is +// itself a thing worth handing to a tester. Said out loud in the setting's +// own description rather than left to be discovered. +var minified = style{ + name: Minified, + prologue: "[", + openID: `{"id":`, + openName: `,"name":"`, + openMail: `","email":"`, + openAmt: `","amount":`, + openAct: `,"active":`, + nullPart: `,"retired":null`, + openTags: `,"tags":["`, + tagSep: `","`, + openAddr: `"],"address":{"city":"`, + openZip: `","zip":"`, + openNote: `"},"note":"`, + tailMore: `"},`, + tailLast: `"}]`, +} + +// indented is what json.dumps(indent=2), JSON.stringify(x, null, 2) and every +// formatter reached for by default produce: two spaces a level, every value on +// its own line, and the nested array and object opened out as well. +// +// Two rather than four, because two is what those three produce without being +// asked. A file indented by four is a different file, and it is a setting of +// its own the day somebody needs it rather than a second value here. +var indented = style{ + name: Indented, + prologue: "[\n", + openID: " {\n \"id\": ", + openName: ",\n \"name\": \"", + openMail: "\",\n \"email\": \"", + openAmt: "\",\n \"amount\": ", + openAct: ",\n \"active\": ", + nullPart: ",\n \"retired\": null", + openTags: ",\n \"tags\": [\n \"", + tagSep: "\",\n \"", + openAddr: "\"\n ],\n \"address\": {\n \"city\": \"", + openZip: "\",\n \"zip\": \"", + openNote: "\"\n },\n \"note\": \"", + tailMore: "\"\n },\n", + tailLast: "\"\n }\n]\n", +} + +var styles = map[string]style{ + RecordPerLine: recordPerLine, + Minified: minified, + Indented: indented, +} + +func defaultStyle() style { return recordPerLine } + +// parseStyle reads the one setting this format takes. +// +// A value outside the declared set has already been refused by the registry, +// which checks every format against its declaration in one place. This branch +// stays for the reason the CSV dialect keeps its own: the function is callable +// directly, a guard is such a caller, and a generator that trusts its input is +// one registry change away from writing a file nobody ordered. +func parseStyle(props map[string]string) (style, error) { + v, ok := props[Formatting] + if !ok || v == "" { + return defaultStyle(), nil + } + s, known := styles[v] + if !known { + return style{}, &format.PropertyValueError{ + Format: "json", Key: Formatting, Value: v, + Reason: "it has to be " + Indented + ", " + Minified + " or " + RecordPerLine, + } + } + return s, nil +} + +func properties() []format.Property { + return []format.Property{ + { + Name: Formatting, Kind: format.PropertyChoice, + Choices: []string{Indented, Minified, RecordPerLine}, + Default: RecordPerLine, + Detail: "How the document is laid out. Every reader accepts all three, so this changes what meets everything around the parser rather than the parser itself - minified puts the whole file on one line and ends without a newline, and indented makes the same records several times larger.", + }, + } +} diff --git a/internal/guard/generatorbytes_test.go b/internal/guard/generatorbytes_test.go index 2ee7104..a275d9f 100644 --- a/internal/guard/generatorbytes_test.go +++ b/internal/guard/generatorbytes_test.go @@ -272,6 +272,13 @@ func goldenCases() map[string]engine.Target { "csv_8kib_seventeen_columns": {ID: "g", Format: "csv", Sizes: engine.Uniform(1, 8192), Label: true, Properties: map[string]string{"columns": "17"}}, "json_8kib": {ID: "g", Format: "json", Sizes: engine.Uniform(1, 8192), Label: true}, + + // The two layouts that are not the default. Whitespace is the whole of + // what changes, and whitespace is exactly what a hash sees. + "json_8kib_minified": {ID: "g", Format: "json", Sizes: engine.Uniform(1, 8192), Label: true, + Properties: map[string]string{"formatting": "minified"}}, + "json_8kib_indented": {ID: "g", Format: "json", Sizes: engine.Uniform(1, 8192), Label: true, + Properties: map[string]string{"formatting": "indented"}}, "xml_8kib": {ID: "g", Format: "xml", Sizes: engine.Uniform(1, 8192), Label: true}, "html_8kib": {ID: "g", Format: "html", Sizes: engine.Uniform(1, 8192), Label: true}, "svg_8kib": {ID: "g", Format: "svg", Sizes: engine.Uniform(1, 8192), Label: true}, diff --git a/internal/guard/jsonlayout_test.go b/internal/guard/jsonlayout_test.go new file mode 100644 index 0000000..9fa02de --- /dev/null +++ b/internal/guard/jsonlayout_test.go @@ -0,0 +1,224 @@ +package guard + +// How a JSON document is laid out, and why that needed a guard of its own. +// +// Measured 2026-09-07 on two implementations in two languages: the same records +// minified, one per line, indented by two and indented by four all parse in +// CPython's json and in V8. So no reader can tell this guard whether the layout +// that was ordered is the layout in the file - which is exactly the question, +// and the reason the structural checker is TOLD the layout rather than left to +// work it out. +// +// The canary at the bottom is the half that makes the rest mean anything: a +// checker told the wrong layout has to refuse. Without it, a checker that +// looked at the setting and shrugged would pass every case above. + +import ( + "bytes" + "context" + "errors" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/donislawdev/TestingFilesGenerator/internal/format" + "github.com/donislawdev/TestingFilesGenerator/internal/format/jsonfile" + "github.com/donislawdev/TestingFilesGenerator/internal/oracle" +) + +var jsonLayouts = []string{jsonfile.Indented, jsonfile.Minified, jsonfile.RecordPerLine} + +func jsonProps(layout string) map[string]string { + return map[string]string{jsonfile.Formatting: layout} +} + +// writeJSONDocument produces one document and insists on the ordered size before +// anything else looks at it. +func writeJSONDocument(t *testing.T, size int64, props map[string]string) []byte { + t.Helper() + d, err := format.Get("json") + if err != nil { + t.Fatal(err) + } + p, err := d.Generator.Plan(format.Request{Bytes: size, Seed: 7741, Label: true, Properties: props}) + if err != nil { + t.Fatalf("planning %d B with %v: %v", size, props, err) + } + var buf bytes.Buffer + if err := d.Generator.Write(context.Background(), &buf, p); err != nil { + t.Fatalf("writing %d B with %v: %v", size, props, err) + } + if int64(buf.Len()) != size { + t.Fatalf("%v: ordered %d B and produced %d - the size is exact or it is an error", + props, size, buf.Len()) + } + return buf.Bytes() +} + +// TestAJSONDocumentIsLaidOutTheWayItWasOrdered is the claim: exact size, a +// layout somebody can see, and a reader that agrees the file is what it says. +// +// The line arithmetic is asserted here as well as in the checker, and that is +// not a duplicate. This one says what each layout IS - minified holds no +// newline at all, a record per line means one line each - in a place where the +// failure names the layout. The checker says the same thing to anybody running +// the tool without Go. +func TestAJSONDocumentIsLaidOutTheWayItWasOrdered(t *testing.T) { + dir := t.TempDir() + d, err := format.Get("json") + if err != nil { + t.Fatal(err) + } + checked, skipped := 0, 0 + + for _, layout := range jsonLayouts { + props := jsonProps(layout) + smallest := d.SmallestAccepted(format.Request{Seed: 7741, Label: true, Properties: props}) + + for _, size := range []int64{smallest, smallest + 1, 4096, 65536} { + t.Run(fmt.Sprintf("%s/%d", layout, size), func(t *testing.T) { + body := writeJSONDocument(t, size, props) + newlines := int64(bytes.Count(body, []byte("\n"))) + + switch layout { + case jsonfile.Minified: + if newlines != 0 { + t.Errorf("minified holds %d newline(s), and minified means none", newlines) + } + if bytes.HasSuffix(body, []byte("\n")) { + t.Error("minified ends with a newline, so it is not minified") + } + default: + if newlines < 3 { + t.Errorf("%s holds %d newline(s), which is not laid out at all", layout, newlines) + } + if !bytes.HasSuffix(body, []byte("]\n")) { + t.Errorf("%s does not close with the array and a newline", layout) + } + } + + path := filepath.Join(dir, fmt.Sprintf("%s-%d.json", layout, size)) + if err := os.WriteFile(path, body, 0o600); err != nil { + t.Fatal(err) + } + res := oracle.Strict("json", path, jsonfile.Formatting+"="+layout) + if !res.Available { + skipped++ + t.Skip("the structural check needs python") + } + if res.Err != nil { + t.Fatalf("%s is not a well formed %s document: %v", path, layout, res.Err) + } + checked++ + }) + } + } + + if checked == 0 { + t.Errorf("nothing was read by anything outside this package - %d case(s) skipped", skipped) + } + t.Logf("%d document(s) read by the structural checker, %d skipped", checked, skipped) +} + +// TestEachJSONLayoutAnswersForItsOwnMinimum holds the floor to the layout. +// +// A layout that opens every value onto its own line needs more room for one +// record than a layout with no whitespace at all, so one declared number +// cannot serve all three. The registry declares the DEFAULT layout's floor and +// the generator answers for the rest, which is how CSV does it - and what makes +// the two agree is that SmallestAccepted asks the generator rather than reading +// the declaration. +func TestEachJSONLayoutAnswersForItsOwnMinimum(t *testing.T) { + d, err := format.Get("json") + if err != nil { + t.Fatal(err) + } + + floors := map[string]int64{} + for _, layout := range jsonLayouts { + props := jsonProps(layout) + floor := d.SmallestAccepted(format.Request{Seed: 7741, Label: true, Properties: props}) + floors[layout] = floor + + if _, err := d.Generator.Plan(format.Request{ + Bytes: floor, Seed: 7741, Label: true, Properties: props}); err != nil { + t.Errorf("%s: the floor it reports is %d B and that is refused: %v", layout, floor, err) + } + + _, err := d.Generator.Plan(format.Request{ + Bytes: floor - 1, Seed: 7741, Label: true, Properties: props}) + var below *format.BelowMinimumError + if !errors.As(err, &below) { + t.Errorf("%s: one byte under the floor was answered with %v, not a BelowMinimumError", layout, err) + continue + } + // The refusal says which layout it is about, because the same size is + // legal in another one and a person reading it needs to know that. + if !strings.Contains(below.Reason, layout) { + t.Errorf("%s: the refusal does not name the layout it is about: %q", layout, below.Reason) + } + } + + // An opened out record cannot need the same room as one with no whitespace. + // Equal floors would mean the layout never reached the arithmetic. + if floors[jsonfile.Indented] <= floors[jsonfile.RecordPerLine] || + floors[jsonfile.RecordPerLine] <= floors[jsonfile.Minified] { + t.Errorf("the floors are %v, and indented has to need more room than a record per line, which needs more than minified", + floors) + } + t.Logf("floors: %v", floors) +} + +// TestTheDefaultLayoutIsTheBytesJSONAlwaysWrote is the way back. +// +// The same pin the text formats got when encoding arrived: saying nothing and +// saying the default out loud have to be one file, because a default that moves +// is a breaking change wearing the clothes of a feature. +func TestTheDefaultLayoutIsTheBytesJSONAlwaysWrote(t *testing.T) { + for _, size := range []int64{219, 1024, 65536} { + silent := writeJSONDocument(t, size, nil) + spoken := writeJSONDocument(t, size, jsonProps(jsonfile.RecordPerLine)) + if !bytes.Equal(silent, spoken) { + t.Errorf("at %d B: saying nothing and saying record-per-line produce different bytes", size) + } + } +} + +// TestTheJSONCheckerRefusesALayoutTheFileIsNot is the canary. +// +// Every case above hands the checker a file and the name of the layout it was +// written in, and a checker that ignored the name would pass all of them. This +// hands it every WRONG name instead. Six pairs, six refusals, or the layer +// underneath is a rubber stamp. +func TestTheJSONCheckerRefusesALayoutTheFileIsNot(t *testing.T) { + dir := t.TempDir() + refused := 0 + + for _, written := range jsonLayouts { + body := writeJSONDocument(t, 4096, jsonProps(written)) + path := filepath.Join(dir, written+".json") + if err := os.WriteFile(path, body, 0o600); err != nil { + t.Fatal(err) + } + for _, told := range jsonLayouts { + if told == written { + continue + } + res := oracle.Strict("json", path, jsonfile.Formatting+"="+told) + if !res.Available { + t.Skip("the structural check needs python") + } + if res.Err == nil { + t.Errorf("a %s document was called a well formed %s one", written, told) + continue + } + refused++ + } + } + + if refused != 6 { + t.Errorf("%d of the 6 wrong pairings were refused", refused) + } +} diff --git a/internal/guard/testdata/generator-golden.json b/internal/guard/testdata/generator-golden.json index cd39848..9a472b4 100644 --- a/internal/guard/testdata/generator-golden.json +++ b/internal/guard/testdata/generator-golden.json @@ -293,6 +293,14 @@ "md_8kib_utf16be": { "bytes": 8192, "sha256": "a2d0cc69cb609277b97ae7bf7e11505fa26f3eb3214e8e62c1eba07e6de7bc24" + }, + "json_8kib_minified": { + "bytes": 8192, + "sha256": "59b4b6d4056adfcd5b914c39b3998a6ed0583992be1c850e9fac083c37a39f7c" + }, + "json_8kib_indented": { + "bytes": 8192, + "sha256": "9b00c22a11836d66cbf653275bb943b299d6864d391cff8ca13fa8ac7bdcc8c9" } }, "remeasured": [ diff --git a/internal/oracle/strict.py b/internal/oracle/strict.py index 4796a45..d83aff6 100644 --- a/internal/oracle/strict.py +++ b/internal/oracle/strict.py @@ -420,14 +420,21 @@ def check_quoting(style, number, value, was_quoted, sep): "no quote and no line break - quote_style minimal quotes only what needs it") -def check_json(data): - """The document is an array of records, each on its own line. +def check_json(data, settings=None): + """The document is an array of records, laid out the way it was ordered. Parsing is CPython's json here and V8's parser in the reference tool beside it, which are two implementations in two languages. What this adds is the shape: that the file is records rather than one enormous value, and that it is laid out the way the manifest says it is. + The layout is TOLD rather than worked out, for the reason the CSV dialect + is told. Until 2026-09-07 there was only one layout and this checked it by + name - "the manifest says one record per line" - which would have called a + minified document broken the moment minified became something a person can + ask for. A checker that guessed instead would agree with any layout it was + handed, including the one nobody ordered. + The value types come from the format document, not from what the generator happens to emit. A generator that quietly stopped writing booleans would be the right size and would still parse. @@ -449,10 +456,28 @@ def check_json(data): if not doc: fail("the array is empty") - # The manifest states one record per line, so the line count has to match. - lines = text.rstrip("\n").split("\n") - if len(lines) != len(doc) + 2: - fail(f"{len(doc)} records over {len(lines)} lines - the manifest says one record per line") + # Each layout has its own arithmetic of lines, and every one of them is + # something a truncated or mis-indented record breaks. + layout = (settings or {}).get("formatting", "record-per-line") + newlines = text.count("\n") + if layout == "minified": + if newlines: + fail(f"minified was ordered and the document holds {newlines} newline(s)") + if text.endswith("\n"): + fail("minified was ordered and the document ends with a newline") + elif layout == "indented": + body = newlines - 2 + if body <= 0 or body % len(doc): + fail(f"{len(doc)} indented records over {newlines} lines, " + f"which is not a whole number of lines each") + per = body // len(doc) + if per < 5: + fail(f"an indented record takes {per} line(s), which is not opened out at all") + elif layout == "record-per-line": + if newlines != len(doc) + 2: + fail(f"{len(doc)} records over {newlines} lines - one record per line was ordered") + else: + fail(f"the json check was told formatting={layout!r}, which is not a layout this tool writes") kinds = set() keys = None @@ -1655,7 +1680,7 @@ def check_md(data, settings=None): # Checks that take the shape of the file as well as its bytes. Everything else # is handed the bytes alone, so adding a setting to one check cannot change how # any other one is called. -TAKES_SETTINGS = {"csv", "txt", "md"} +TAKES_SETTINGS = {"csv", "txt", "md", "json"} if __name__ == "__main__": if len(sys.argv) < 3 or sys.argv[1] not in CHECKS: diff --git a/web/public/formats/index.html b/web/public/formats/index.html index 30cceb5..6e30225 100644 --- a/web/public/formats/index.html +++ b/web/public/formats/index.html @@ -415,6 +415,11 @@

Settings each format accepts

quality 1 - 100 + + json + formatting + indented, minified, record-per-line + jxl width diff --git a/web/public/pl/formaty/index.html b/web/public/pl/formaty/index.html index 8a9f235..2ed90a6 100644 --- a/web/public/pl/formaty/index.html +++ b/web/public/pl/formaty/index.html @@ -415,6 +415,11 @@

Ustawienia, które przyjmuje każdy format

quality 1 - 100 + + json + formatting + indented, minified, record-per-line + jxl width From 4f140720a9b1c49984c1622703e3ea5dab2a43d6 Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Mon, 7 Sep 2026 21:22:03 +0200 Subject: [PATCH 2/2] fix: say out loud that the window offers the JSON layout setting The engine gained property:json.formatting and neither parity list named it, so TestEveryEngineCapabilityIsClassifiedForBothSurfaces refused the run on all four jobs. That is the guard doing its job: a setting reaching the engine without anybody answering whether the window offers it is exactly the drift D1 forbids. The bar for the reachable list is two things rather than one - a control on the screen, and a guard that presses it and finds the value on the other side - and both were measured before the name went in. The field is drawn from the declaration and nothing else, the menu opens on its declared default and is named among the twenty that do, and the path from a field to the manifest is pinned by the guard that types a value and reads it back off the disk. One thing was worth asking rather than assuming. The window sends every setting, because a menu cannot be empty, so a run started there says record-per-line out loud where the command line says nothing at all. Those two have to be one file, and the guard that pins them already exists. D1 parity: 114 of 125 capabilities reachable from the window, eleven still to go. Co-Authored-By: Claude Opus 5 --- internal/guard/parity_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/internal/guard/parity_test.go b/internal/guard/parity_test.go index 260df6a..b7bfeb6 100644 --- a/internal/guard/parity_test.go +++ b/internal/guard/parity_test.go @@ -118,6 +118,7 @@ var reachableFromTheWindow = []string{ "property:jpg.height", "property:jpg.quality", "property:jpg.width", + "property:json.formatting", "property:log.entry_format", "property:log.ip_version", "property:log.level_mix",