diff --git a/COMPARISON.md b/COMPARISON.md index 75cc5ed5..ef5bc99d 100644 --- a/COMPARISON.md +++ b/COMPARISON.md @@ -22,7 +22,7 @@ Scope: how the [documents.js ecosystem](README.md) compares against real alterna **documents.js counterpart:** byte-codec, archive-codec -This is the category where documents.js's 'hand-written, dependency-minimal' claim is least differentiating — essentially every alternative is also hand-written or ported-from-C with no dependencies. The real question byte-codec has to answer is why it exists rather than depending on fflate. For archive-codec, the closest true counterpart is SheetJS's cfb (same MS-CFB target, plus limited writing) — but cfb has no ZIP awareness, and no ZIP library has OLE/CFB support, so the combination of recursive ZIP-in-ZIP walking with depth/size guards alongside bounded CFB reading in one package is genuinely unmatched, though narrow. +This is the category where documents.js's 'hand-written, dependency-minimal' claim is least differentiating — essentially every alternative is also hand-written or ported-from-C with no dependencies. The real question byte-codec has to answer is why it exists rather than depending on fflate. For archive-codec, the closest true counterpart is SheetJS's cfb, which targets MS-CFB in both directions as archive-codec now does — but cfb has no ZIP awareness, and no ZIP library has OLE/CFB support, so the combination of recursive ZIP-in-ZIP walking with depth/size guards alongside bounded CFB reading and writing in one package is genuinely unmatched, though narrow. | Package / service | Direction | Approach | Deployment | Licence | Pricing model | Status | | -------------------------------------------------------------- | ------------ | ------------ | ---------- | ------------------------- | ------------------ | --------- | @@ -116,7 +116,7 @@ Standard CRC-32/CRC-32C checksum implementation with a bundled CLI. (npm · v1.2 Pure-JS MS-CFB (classic OLE) container reader/writer, part of SheetJS. (npm · v1.2.2 · 2022-04-06) -**vs. documents.js:** Closest direct counterpart to archive-codec's CFB reader — same approach, same target — and goes further with limited CFB writing, but has zero ZIP awareness. +**vs. documents.js:** Closest direct counterpart to archive-codec's CFB support — same approach, same target, and the same two directions now that archive-codec writes compound files as well as reading them — but has zero ZIP awareness. **Free tier:** N/A — the package itself is free and open source; no paid tiers exist. diff --git a/README.md b/README.md index c258b787..400f804c 100644 --- a/README.md +++ b/README.md @@ -18,13 +18,13 @@ The packages layer from foundation up to user-facing interfaces. Each depends on ### Foundation -| Package | What it is | -| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`document-schema.js`](packages/document-schema.js/README.md) | The canonical, format-agnostic content and document-tree schema shared by every codec, plus the structural transform between them (`decompose`/`flattenTree`/`factorStyles`/`assembleTree`, converting a flat `ContentDocument` to and from the tree-form `DocumentTree`). Free of any format-specific or I/O behaviour: the transform lives here because every codec depends on this package and none of them depends on `documents.js`, so it is the only layer a codec can reach to expose `DocumentTree` publicly without a dependency cycle. | -| [`byte-codec`](packages/byte-codec/README.md) | Generic byte-level primitives (`ByteWriter`, `ByteReader`, CRC-32, deflate/inflate) and PNG/JPEG image encoding and decoding, with zero knowledge of any document format. | -| [`document-outline.js`](packages/document-outline.js/README.md) | Utilities for consumers holding a tree-form `DocumentTree`: the TOC outline projection, effective-property resolution, and the flatten/leaf-text/stable-hash helpers. Depends on the schema alone, and is consumed by the interface packages rather than by the codecs. | -| [`archive-codec`](packages/archive-codec/README.md) | Recursive archive (ZIP-in-ZIP) detection and walking with depth and cumulative decompressed-size guards, plus bounded classic OLE compound-file ([MS-CFB]) reading and OLE Package stream unwrapping — zero document-format knowledge. Consumed by `ooxml.js`, whose pptx (`p:oleObj`) and docx (`o:OLEObject`) OLE reading detects a ZIP-payload embedded object through it and decodes the nested package as a content document, and unwraps the classic `.bin` compound-file spelling through its CFB reader to the same nested decode. | -| [`document-compute.js`](packages/document-compute.js/README.md) | Units-typed evaluation over the schema's `MathExpression`: `evaluate()` for point values and bounded intervals through one interpreter with exact-rational unit conversion, plus `solveFor()` numeric root-finding on one unknown. Depends on the schema alone; a leaf nothing else depends on yet — it is not wired into any conversion pipeline. | +| Package | What it is | +| --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [`document-schema.js`](packages/document-schema.js/README.md) | The canonical, format-agnostic content and document-tree schema shared by every codec, plus the structural transform between them (`decompose`/`flattenTree`/`factorStyles`/`assembleTree`, converting a flat `ContentDocument` to and from the tree-form `DocumentTree`). Free of any format-specific or I/O behaviour: the transform lives here because every codec depends on this package and none of them depends on `documents.js`, so it is the only layer a codec can reach to expose `DocumentTree` publicly without a dependency cycle. | +| [`byte-codec`](packages/byte-codec/README.md) | Generic byte-level primitives (`ByteWriter`, `ByteReader`, CRC-32, deflate/inflate) and PNG/JPEG image encoding and decoding, with zero knowledge of any document format. | +| [`document-outline.js`](packages/document-outline.js/README.md) | Utilities for consumers holding a tree-form `DocumentTree`: the TOC outline projection, effective-property resolution, and the flatten/leaf-text/stable-hash helpers. Depends on the schema alone, and is consumed by the interface packages rather than by the codecs. | +| [`archive-codec`](packages/archive-codec/README.md) | Recursive archive (ZIP-in-ZIP) detection and walking with depth and cumulative decompressed-size guards, plus bounded classic OLE compound-file ([MS-CFB]) reading, conformant [MS-CFB] writing, and OLE Package stream unwrapping — zero document-format knowledge. Consumed by `ooxml.js`, whose pptx (`p:oleObj`) and docx (`o:OLEObject`) OLE reading detects a ZIP-payload embedded object through it and decodes the nested package as a content document, and unwraps the classic `.bin` compound-file spelling through its CFB reader to the same nested decode; the writer is the container the legacy binary codecs need before any of them can gain a write path. | +| [`document-compute.js`](packages/document-compute.js/README.md) | Units-typed evaluation over the schema's `MathExpression`: `evaluate()` for point values and bounded intervals through one interpreter with exact-rational unit conversion, plus `solveFor()` numeric root-finding on one unknown. Depends on the schema alone; a leaf nothing else depends on yet — it is not wired into any conversion pipeline. | ### Format codecs diff --git a/packages/archive-codec/README.md b/packages/archive-codec/README.md index 042ee260..c666d956 100644 --- a/packages/archive-codec/README.md +++ b/packages/archive-codec/README.md @@ -2,13 +2,15 @@ [![GitHub](https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white)](https://github.com/ExaDev/documents.js/tree/main/packages/archive-codec) [![npm](https://img.shields.io/badge/npm-CB3837?logo=npm&logoColor=white)](https://www.npmjs.com/package/archive-codec) [![npm version](https://img.shields.io/npm/v/archive-codec)](https://www.npmjs.com/package/archive-codec) [![CI](https://img.shields.io/github/actions/workflow/status/ExaDev/documents.js/ci.yml?branch=main)](https://github.com/ExaDev/documents.js/actions) -> ZIP-in-ZIP recursive walking under depth and cumulative decompressed-size guards, and bounded classic OLE compound-file ([MS-CFB]) reading — zero document-format knowledge, the archive and container utility package for the [documents.js family](../../README.md). Worker-isomorphic: the same code runs under Node and inside a Cloudflare Workers isolate. +> ZIP-in-ZIP recursive walking under depth and cumulative decompressed-size guards, and classic OLE compound-file ([MS-CFB]) reading and writing — zero document-format knowledge, the archive and container utility package for the [documents.js family](../../README.md). Worker-isomorphic: the same code runs under Node and inside a Cloudflare Workers isolate. Created for [documents.js#564](https://github.com/ExaDev/documents.js/issues/564): nothing in the ecosystem recursed into a nested archive. Most concretely, OOXML's embedded-object model — a docx/pptx carrying a genuinely separate ZIP blob at `word/embeddings/oleObject1.xlsx` — had no safe handling anywhere, and no package guarded against recursive-archive inputs at all (`byte-codec`'s 512 MiB per-stream inflate cap does not compose across recursion). A new sibling was chosen over extending `byte-codec` (whose charter is byte/image primitives, zero container-format knowledge) or doing it inline in `documents.js` (which would repeat the duplication `byte-codec`'s own extraction was meant to avoid). Its first family consumer is `ooxml.js`'s OLE embedded-object recovery — [documents.js#733](https://github.com/ExaDev/documents.js/issues/733) (pptx, `p:oleObj`) and [documents.js#734](https://github.com/ExaDev/documents.js/issues/734) (docx, `o:OLEObject`): an OLE payload part's bytes are checked through `isZipArchive` and, when they are a ZIP, decoded as a nested OOXML package behind this package's guarded walk — the bounded inflate that populates `document-schema.js`'s `ContentEmbeddedObject`/`ContentEmbeddedObjectBlock` (the same vocabulary odf.js embeds formula sub-documents through) with a genuinely recovered sub-document. [documents.js#739](https://github.com/ExaDev/documents.js/issues/739) widened the charter from that ZIP-only v1 scope to the classic OLE compound file, recording the decision explicitly rather than by accident (mirroring the #564 reasoning): real-world Word and PowerPoint files frequently store the embeddee as a `.bin` compound file at `word|ppt/embeddings/oleObject1.bin`, and a CFB reader is container knowledge exactly the way ZIP structure is — sectors, FAT chains, and directory entries, never that any stream is a document. The same recovery now unwraps such a payload's `Package` stream ([MS-OLEDS]'s OLE packaging of the real file) through this package and feeds the packaged ZIP to the unchanged nested decode. -Scope: **ZIP containers** (read and write over [`fflate`](https://github.com/101arrowz/fflate), recursive walking of ZIP-in-ZIP entries) and **classic OLE compound files** (bounded [MS-CFB] reading, plus the OLE Package stream unwrapping). **tar and gzip are explicitly out of scope.** +[documents.js#815](https://github.com/ExaDev/documents.js/issues/815), [#816](https://github.com/ExaDev/documents.js/issues/816), and [#817](https://github.com/ExaDev/documents.js/issues/817) then needed the other direction. `xls-codec`, `doc-codec`, `ppt-codec`, and `wpd-codec` each read a legacy Office binary format out of an [MS-CFB] container, and none of them can write one back, because there was no container to put their streams into: a `.xls` writer producing a `Workbook` stream, or a `.doc` writer producing `WordDocument` and `1Table`, needs a conformant compound file to hold them. That container is structural knowledge exactly as the reader's is, so `writeCompoundFile` is the mirror of `readCompoundFile` here rather than four hand-rolled emitters in four codecs. + +Scope: **ZIP containers** (read and write over [`fflate`](https://github.com/101arrowz/fflate), recursive walking of ZIP-in-ZIP entries) and **classic OLE compound files** (bounded [MS-CFB] reading and conformant [MS-CFB] writing, plus the OLE Package stream unwrapping). **tar and gzip are explicitly out of scope.** ## Getting started @@ -38,14 +40,15 @@ import { walkArchive } from "archive-codec/zip/walk"; The smoke suite (`test/smoke.test.mjs`) is the guard on that advertisement: it loads each module below from the built `dist/` in both module systems, so a build config that stops serving an advertised subpath fails the suite — neither publint nor `attw` catches a wildcard whose targets are missing. -| Module | Exports | -| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `zip/container` | `zipPackage` (ordered-entries ZIP write with stored-uncompressed support), `unzipPackage`, `ZipEntry` | -| `zip/detect` | `detectArchiveFormat` (`'zip' \| 'cfb' \| 'unknown'`), `isZipArchive`, `ArchiveFormat` | -| `zip/walk` | `walkArchive` (recursive ZIP-in-ZIP walking), `ArchiveWalkEntry`, `ArchiveWalkLimitError`, `MAX_WALK_DEPTH`, `MAX_WALK_TOTAL_BYTES`, `WalkArchiveOptions` | -| `cfb/detect` | `isCompoundFile` (the `D0 CF 11 E0 …` magic-byte check) | -| `cfb/read` | `readCompoundFile` (bounded [MS-CFB] stream extraction), `CompoundFileStream`, `CompoundFileFormatError`, `MAX_CFB_TOTAL_STREAM_BYTES`, `ReadCompoundFileOptions` | -| `cfb/ole-package` | `readOlePackage` (OLE Package stream unwrapping), `OlePackage`, `OlePackageFormatError` | +| Module | Exports | +| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `zip/container` | `zipPackage` (ordered-entries ZIP write with stored-uncompressed support), `unzipPackage`, `ZipEntry` | +| `zip/detect` | `detectArchiveFormat` (`'zip' \| 'cfb' \| 'unknown'`), `isZipArchive`, `ArchiveFormat` | +| `zip/walk` | `walkArchive` (recursive ZIP-in-ZIP walking), `ArchiveWalkEntry`, `ArchiveWalkLimitError`, `MAX_WALK_DEPTH`, `MAX_WALK_TOTAL_BYTES`, `WalkArchiveOptions` | +| `cfb/detect` | `isCompoundFile` (the `D0 CF 11 E0 …` magic-byte check) | +| `cfb/read` | `readCompoundFile` (bounded [MS-CFB] stream extraction), `CompoundFileStream`, `CompoundFileFormatError`, `MAX_CFB_TOTAL_STREAM_BYTES`, `ReadCompoundFileOptions` | +| `cfb/write` | `writeCompoundFile` ([MS-CFB] container generation), `CompoundFileWriteError`, `WriteCompoundFileOptions` — takes the `CompoundFileStream` array `cfb/read` returns | +| `cfb/ole-package` | `readOlePackage` (OLE Package stream unwrapping), `OlePackage`, `OlePackageFormatError` | ### Recursive walking @@ -88,6 +91,28 @@ if (packageStream !== undefined) { Reading is bounded the same way walking is: chain cycles and out-of-range sectors fail against bounds derived from the file's own sector count, and one cumulative extracted-bytes budget (`MAX_CFB_TOTAL_STREAM_BYTES`, 512 MiB — the same figure the family grants one decompressed stream) bounds the multiplication a hostile FAT gains by aliasing one sector into many streams. Every structural failure throws rather than truncating — a malformed compound file fails whole, never a partial stream listing that looks complete. Version 3 (512-byte sectors) and version 4 (4096-byte) files both read; the mini-FAT path every stream shorter than the header's cutoff takes is first-class, because a small real-world embed genuinely lands there. +Writing is the mirror image, taking the same `CompoundFileStream` array reading returns: + +```ts +import { readCompoundFile, writeCompoundFile } from "archive-codec"; + +const bytes = writeCompoundFile([ + { path: "WordDocument", bytes: mainStream }, + { path: "1Table", bytes: tableStream }, + { path: "SummaryInformation", bytes: summaryStream }, + { path: "ObjectPool/_1234/Package", bytes: embeddedFile }, // a nested storage +]); + +// ... so re-writing what was read is a round trip, not a translation. +writeCompoundFile(readCompoundFile(bytes)); +``` + +Slash-separated paths name the enclosing storages exactly as reading reports them, so nested storages are written as well as read; a request that cannot be expressed as a conformant file throws `CompoundFileWriteError` rather than producing bytes that only look valid — an over-long or illegally named entry (`\`, `:`, and `!` are the characters [MS-CFB] forbids), an empty path segment, two siblings whose names collide under the format's case-insensitive ordering, or a version 3 stream past the 2 GB the format allows one. Both allocation paths are written: a stream at or above the 4096-byte cutoff takes FAT-chained sectors, one below it a run of 64-byte mini sectors in the root entry's own mini stream. Files past the 6.875 MiB that the header's own 109-entry DIFAT array can address spill into chained DIFAT sectors rather than failing, which matters because a real `.doc` or `.xls` reaches that size routinely. + +Two details are deliberate rather than incidental. The directory's sibling trees are genuine red-black trees — balanced by construction and coloured so that every [MS-CFB] 2.6.4 constraint holds, including the black-height property — because the sibling tree exists to be binary-searched by name, and the degenerate right-sibling chain that a purely structural reader would still accept is not a search tree. And the output depends only on the set of paths, never on the order they were supplied in, since the directory's order is the format's own name ordering: two callers building the same file from differently ordered lists get identical bytes. + +Correctness is checked against independent parsers, not only against this package's own reader: the written files are accepted by [`olefile`](https://github.com/decalage2/olefile) in its strict `DEFECT_INCORRECT` mode and by 7-Zip's Compound handler, both of which return byte-identical stream content, and a real LibreOffice-authored `.doc` read through `readCompoundFile` and re-emitted through `writeCompoundFile` still opens in LibreOffice Writer. + ### ZIP container `zipPackage` takes an _ordered_ array of `[path, entry]` tuples, not a `Record`, so the caller controls the exact emission order deterministically (the property formats with a fixed-offset first entry — ODF's `mimetype` — depend on), and any entry can be written stored-uncompressed via `stored: true`. `unzipPackage` is the read side; the returned `Record` makes no ordering promise and collapses duplicate paths. diff --git a/packages/archive-codec/src/cfb/read.ts b/packages/archive-codec/src/cfb/read.ts index 2fc1042e..f246d4c0 100644 --- a/packages/archive-codec/src/cfb/read.ts +++ b/packages/archive-codec/src/cfb/read.ts @@ -28,7 +28,7 @@ export class CompoundFileFormatError extends Error { } } -// One extracted stream: its path is the slash-joined names of the storages enclosing it plus its own name, root-relative with no leading slash (a root-level stream's path is just its name -- the OLE packaging's "Package" stream reads back as 'Package'). +// One named stream: its path is the slash-joined names of the storages enclosing it plus its own name, root-relative with no leading slash (a root-level stream's path is just its name -- the OLE packaging's "Package" stream reads back as 'Package'). This is the package's compound-file vocabulary in both directions, not the reader's alone: writeCompoundFile (src/cfb/write.ts) takes the same array this returns, so re-writing what was read is a well-typed round trip rather than a translation between two shapes. export interface CompoundFileStream { readonly path: string; readonly bytes: Uint8Array; diff --git a/packages/archive-codec/src/cfb/write.test.ts b/packages/archive-codec/src/cfb/write.test.ts new file mode 100644 index 00000000..01d923a3 --- /dev/null +++ b/packages/archive-codec/src/cfb/write.test.ts @@ -0,0 +1,640 @@ +import { describe, expect, it } from "vitest"; +import { type CompoundFileStream, readCompoundFile } from "./read"; +import { CompoundFileWriteError, writeCompoundFile } from "./write"; + +// Coverage for the [MS-CFB] writer (src/cfb/write.ts). Two kinds of check, deliberately kept separate: +// +// 1. Byte-layout assertions derived by hand from the spec's own field tables ([MS-CFB] 2.2 header, 2.3 FAT, 2.4 mini FAT, 2.5 DIFAT, 2.6.1 directory entry), so a header field silently written at the wrong offset or in the wrong endianness fails here rather than surviving because this package's own reader happens to make the same mistake. +// 2. Round trips through readCompoundFile, which is the real correctness proof for anything structural: chains, the mini stream, nested storages, and the DIFAT chain are all things a hand-checked byte dump cannot practically cover. +// +// The red-black invariants get their own directory-parsing check, because they are the one part of the format where a wrong-but-plausible answer (insertion order, or an unbalanced right-spine chain) reads back perfectly through a structural reader and is still rejected by a reader that validates the tree. + +const enc = (s: string): Uint8Array => new TextEncoder().encode(s); + +const stream = ( + path: string, + bytes: Uint8Array, +): CompoundFileStream => ({ path, bytes }); + +const u16 = (bytes: Uint8Array, offset: number): number => + new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength).getUint16( + offset, + true, + ); + +const u32 = (bytes: Uint8Array, offset: number): number => + new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength).getUint32( + offset, + true, + ); + +const ENDOFCHAIN = 0xfffffffe; +const FREESECT = 0xffffffff; +const FATSECT = 0xfffffffd; +const DIFSECT = 0xfffffffc; +const NOSTREAM = 0xffffffff; + +// Byte-for-byte comparison that reports the first differing index rather than dumping megabytes into the failure message, for the multi-megabyte fixtures the FAT- and DIFAT-growth tests need. +function firstDifference( + actual: Uint8Array, + expected: Uint8Array, +): number { + if (actual.length !== expected.length) { + return Math.min(actual.length, expected.length); + } + for (let i = 0; i < actual.length; i++) { + if (actual[i] !== expected[i]) { + return i; + } + } + return -1; +} + +function expectSameBytes( + actual: Uint8Array | undefined, + expected: Uint8Array, +): void { + expect(actual?.length).toBe(expected.length); + expect(firstDifference(actual ?? new Uint8Array(0), expected)).toBe(-1); +} + +interface ParsedDirectoryEntry { + readonly id: number; + readonly name: string; + readonly nameLength: number; + readonly objectType: number; + readonly colour: number; + readonly left: number; + readonly right: number; + readonly child: number; + readonly startSector: number; + readonly size: number; +} + +// An independent directory reader for the invariant checks below: header -> header DIFAT -> FAT -> directory chain. Deliberately not readCompoundFile, which returns extracted streams and discards the entry links, colours, and sibling structure this file exists to assert on. Only the header's own 109 DIFAT entries are followed, which covers every fixture here that inspects entries. +function parseDirectory( + bytes: Uint8Array, +): ParsedDirectoryEntry[] { + const sectorSize = 1 << u16(bytes, 0x1e); + const sectorAt = (sector: number): number => (sector + 1) * sectorSize; + const fatSectorCount = u32(bytes, 0x2c); + const fatSectors: number[] = []; + for (let i = 0; i < Math.min(fatSectorCount, 109); i++) { + fatSectors.push(u32(bytes, 0x4c + i * 4)); + } + const fatEntry = (sector: number): number => { + const perSector = sectorSize / 4; + const holder = fatSectors[Math.floor(sector / perSector)]; + if (holder === undefined) { + throw new Error( + `test directory parse: no FAT sector holds the entry for sector ${sector}`, + ); + } + return u32(bytes, sectorAt(holder) + (sector % perSector) * 4); + }; + + const directory: number[] = []; + for ( + let sector = u32(bytes, 0x30); + sector !== ENDOFCHAIN; + sector = fatEntry(sector) + ) { + directory.push(sector); + if (directory.length > 1024) { + throw new Error( + "test directory parse: directory chain did not terminate", + ); + } + } + + const entries: ParsedDirectoryEntry[] = []; + const perDirectorySector = sectorSize / 128; + for (let i = 0; i < directory.length; i++) { + const sector = directory[i] ?? 0; + for (let slot = 0; slot < perDirectorySector; slot++) { + const base = sectorAt(sector) + slot * 128; + const nameLength = u16(bytes, base + 0x40); + entries.push({ + id: i * perDirectorySector + slot, + name: new TextDecoder("utf-16le").decode( + bytes.subarray(base, base + Math.max(0, nameLength - 2)), + ), + nameLength, + objectType: bytes[base + 0x42] ?? 0, + colour: bytes[base + 0x43] ?? 0, + left: u32(bytes, base + 0x44), + right: u32(bytes, base + 0x48), + child: u32(bytes, base + 0x4c), + startSector: u32(bytes, base + 0x74), + size: u32(bytes, base + 0x78) + u32(bytes, base + 0x7c) * 4294967296, + }); + } + } + return entries; +} + +// The [MS-CFB] 2.6.4 sorting relationship, restated independently of the implementation: a shorter name is less than a longer one, and equal-length names compare by uppercased UTF-16 code point. Every name in these fixtures is ASCII, so the uppercase mapping here is the plain one. +function compareNamesForTest(a: string, b: string): number { + if (a.length !== b.length) { + return a.length - b.length; + } + const ua = a.toUpperCase(); + const ub = b.toUpperCase(); + return ua < ub ? -1 : ua > ub ? 1 : 0; +} + +// Asserts every [MS-CFB] 2.6.4 constraint over one storage's sibling tree, plus the black-height property a red-black tree carries by definition, and returns the tree's black height so a caller can recurse into child storages. +function expectRedBlackTree( + entries: readonly ParsedDirectoryEntry[], + rootId: number, +): void { + const entryOf = (id: number): ParsedDirectoryEntry => { + const entry = entries[id]; + if (entry === undefined) { + throw new Error( + `sibling tree links to entry ${id}, which the directory does not hold`, + ); + } + return entry; + }; + + if (rootId === NOSTREAM) { + return; + } + // Constraint 1: the top of each sibling tree is black. + expect(entryOf(rootId).colour).toBe(1); + + const blackHeight = (id: number, parentColour: number): number => { + if (id === NOSTREAM) { + return 1; + } + const entry = entryOf(id); + expect(entry.colour === 0 || entry.colour === 1).toBe(true); + // Constraint 2: two consecutive nodes must not both be red. + if (entry.colour === 0) { + expect(parentColour).toBe(1); + } + // Constraint 3: the left sibling is less than the right sibling, which over the whole tree means it is a genuine binary search tree under the spec's ordering. + if (entry.left !== NOSTREAM) { + expect( + compareNamesForTest(entryOf(entry.left).name, entry.name), + ).toBeLessThan(0); + } + if (entry.right !== NOSTREAM) { + expect( + compareNamesForTest(entryOf(entry.right).name, entry.name), + ).toBeGreaterThan(0); + } + const left = blackHeight(entry.left, entry.colour); + const right = blackHeight(entry.right, entry.colour); + // Every path from a node down to a leaf holds the same number of black nodes -- the defining red-black property, and the one a chain of right siblings (the naive "sorted linked list" shape) fails. + expect(left).toBe(right); + return left + (entry.colour === 1 ? 1 : 0); + }; + blackHeight(rootId, 1); +} + +describe("writeCompoundFile header and sector layout", () => { + // One 5-byte stream: small enough for the mini stream, so the file is the minimal shape that still exercises every structure -- header, one FAT sector, one directory sector, the mini stream, and the mini FAT. Every expectation below is derived from the spec's field tables, then checked against the layout this writer commits to: sector 0 FAT, sector 1 directory, sector 2 mini stream, sector 3 mini FAT. + const minimal = writeCompoundFile([stream("Foo", enc("hello"))]); + + it("writes the header signature, CLSID, versions, and byte order [MS-CFB] 2.2", () => { + expect([...minimal.subarray(0, 8)]).toEqual([ + 0xd0, 0xcf, 0x11, 0xe0, 0xa1, 0xb1, 0x1a, 0xe1, + ]); + // Header CLSID MUST be all zeroes (CLSID_NULL). + expect([...minimal.subarray(0x08, 0x18)]).toEqual( + Array.from({ length: 16 }, () => 0), + ); + expect(u16(minimal, 0x18)).toBe(0x003e); // minor version, SHOULD be 0x003E for major 3 or 4 + expect(u16(minimal, 0x1a)).toBe(3); + expect(u16(minimal, 0x1c)).toBe(0xfffe); // byte order mark: little-endian + expect(u16(minimal, 0x1e)).toBe(9); // sector shift: 2^9 = 512, mandated for major version 3 + expect(u16(minimal, 0x20)).toBe(6); // mini sector shift: 2^6 = 64 + // Reserved (6 bytes) MUST be all zeroes. + expect([...minimal.subarray(0x22, 0x28)]).toEqual([0, 0, 0, 0, 0, 0]); + }); + + it("writes the sector-count and location fields the minimal file's layout implies", () => { + expect(u32(minimal, 0x28)).toBe(0); // number of directory sectors MUST be zero for major version 3 + expect(u32(minimal, 0x2c)).toBe(1); // one FAT sector maps all four sectors of this file + expect(u32(minimal, 0x30)).toBe(1); // first directory sector + expect(u32(minimal, 0x34)).toBe(0); // transaction signature number, zero with no transaction support + expect(u32(minimal, 0x38)).toBe(0x1000); // mini stream cutoff MUST be 0x00001000 + expect(u32(minimal, 0x3c)).toBe(3); // first mini FAT sector + expect(u32(minimal, 0x40)).toBe(1); // one mini FAT sector + expect(u32(minimal, 0x44)).toBe(ENDOFCHAIN); // no DIFAT sector is needed, so the chain is empty + expect(u32(minimal, 0x48)).toBe(0); + }); + + it("writes the header DIFAT array: the FAT sector locations in order, then FREESECT padding", () => { + expect(u32(minimal, 0x4c)).toBe(0); + for (let i = 1; i < 109; i++) { + expect(u32(minimal, 0x4c + i * 4)).toBe(FREESECT); + } + }); + + it("sizes the file at one header sector plus its sectors, and lays the sectors out contiguously", () => { + // Sector N occupies bytes [(N + 1) * sectorSize, (N + 2) * sectorSize) ([MS-CFB] 2.3), so a four-sector version 3 file is 5 * 512 bytes. + expect(minimal.length).toBe(512 * 5); + }); + + it("marks the FAT sector as FATSECT and terminates every one-sector chain [MS-CFB] 2.3", () => { + const fat = 512; + expect(u32(minimal, fat + 0 * 4)).toBe(FATSECT); // sector 0 holds the FAT itself + expect(u32(minimal, fat + 1 * 4)).toBe(ENDOFCHAIN); // directory + expect(u32(minimal, fat + 2 * 4)).toBe(ENDOFCHAIN); // mini stream + expect(u32(minimal, fat + 3 * 4)).toBe(ENDOFCHAIN); // mini FAT + // Entries covering past the end of the file MUST be FREESECT. + for (let i = 4; i < 128; i++) { + expect(u32(minimal, fat + i * 4)).toBe(FREESECT); + } + }); + + it("writes the root directory entry per [MS-CFB] 2.6.1/2.6.2", () => { + const root = 512 * 2; + expect( + new TextDecoder("utf-16le").decode(minimal.subarray(root, root + 20)), + ).toBe("Root Entry"); + expect(u16(minimal, root + 0x40)).toBe(22); // 10 code points plus the terminating null, doubled + expect(minimal[root + 0x42]).toBe(5); // root storage object + expect(minimal[root + 0x43]).toBe(1); // the root storage object MUST always be black + expect(u32(minimal, root + 0x44)).toBe(NOSTREAM); + expect(u32(minimal, root + 0x48)).toBe(NOSTREAM); + expect(u32(minimal, root + 0x4c)).toBe(1); // its only child is the sole stream entry + expect([...minimal.subarray(root + 0x50, root + 0x60)]).toEqual( + Array.from({ length: 16 }, () => 0), + ); // CLSID + expect(u32(minimal, root + 0x60)).toBe(0); // state bits + expect([...minimal.subarray(root + 0x64, root + 0x74)]).toEqual( + Array.from({ length: 16 }, () => 0), + ); // creation and modified time MUST be zero for the root + expect(u32(minimal, root + 0x74)).toBe(2); // the mini stream's first sector + expect(u32(minimal, root + 0x78)).toBe(64); // the mini stream is one 64-byte mini sector long + expect(u32(minimal, root + 0x7c)).toBe(0); + }); + + it("writes the stream directory entry, mini-resident because it is under the cutoff", () => { + const entry = 512 * 2 + 128; + expect( + new TextDecoder("utf-16le").decode(minimal.subarray(entry, entry + 6)), + ).toBe("Foo"); + expect(u16(minimal, entry + 0x40)).toBe(8); + expect(minimal[entry + 0x42]).toBe(2); // stream object + expect(u32(minimal, entry + 0x44)).toBe(NOSTREAM); + expect(u32(minimal, entry + 0x48)).toBe(NOSTREAM); + expect(u32(minimal, entry + 0x4c)).toBe(NOSTREAM); // a stream entry MUST have no child + expect(u32(minimal, entry + 0x74)).toBe(0); // mini sector 0 of the mini stream + expect(u32(minimal, entry + 0x78)).toBe(5); + expect(u32(minimal, entry + 0x7c)).toBe(0); + }); + + it("writes the unallocated directory entries padding the sector as object type 0 with NOSTREAM links", () => { + for (const slot of [2, 3]) { + const base = 512 * 2 + slot * 128; + expect(u16(minimal, base + 0x40)).toBe(0); + expect(minimal[base + 0x42]).toBe(0); // unknown or unallocated + expect(u32(minimal, base + 0x44)).toBe(NOSTREAM); + expect(u32(minimal, base + 0x48)).toBe(NOSTREAM); + expect(u32(minimal, base + 0x4c)).toBe(NOSTREAM); + } + }); + + it("stores the small stream in the mini stream, zero-padded to a whole mini sector, chained by the mini FAT", () => { + const miniStream = 512 * 3; + expect([...minimal.subarray(miniStream, miniStream + 5)]).toEqual([ + ...enc("hello"), + ]); + expect([...minimal.subarray(miniStream + 5, miniStream + 64)]).toEqual( + Array.from({ length: 59 }, () => 0), + ); + const miniFat = 512 * 4; + expect(u32(minimal, miniFat)).toBe(ENDOFCHAIN); + for (let i = 1; i < 128; i++) { + expect(u32(minimal, miniFat + i * 4)).toBe(FREESECT); + } + }); + + it("zero-pads a version 4 header out to its full 4096-byte sector [MS-CFB] 2.2", () => { + const bytes = writeCompoundFile([stream("Foo", enc("hello"))], { + majorVersion: 4, + }); + expect(u16(bytes, 0x1a)).toBe(4); + expect(u16(bytes, 0x1e)).toBe(12); // sector shift: 2^12 = 4096, mandated for major version 4 + expect(u32(bytes, 0x28)).toBe(1); // the directory-sector count is carried for version 4, unlike version 3 + expect([...bytes.subarray(512, 4096)]).toEqual( + Array.from({ length: 3584 }, () => 0), + ); + expect(bytes.length % 4096).toBe(0); + }); +}); + +describe("writeCompoundFile round-trips through readCompoundFile", () => { + it("round-trips a mini-stream-resident stream shorter than the cutoff", () => { + const payload = enc("a small stream"); + const streams = readCompoundFile( + writeCompoundFile([stream("Small", payload)]), + ); + expect(streams.map((s) => s.path)).toEqual(["Small"]); + expectSameBytes(streams[0]?.bytes, payload); + }); + + it("round-trips a FAT-resident stream at exactly the cutoff", () => { + // 4096 is >= the cutoff, so the stream is allocated from the FAT rather than the mini FAT ([MS-CFB] 2.2) -- the exact boundary the comparison has to get right. + const payload = enc("B".repeat(4096)); + const streams = readCompoundFile( + writeCompoundFile([stream("AtCutoff", payload)]), + ); + expectSameBytes(streams[0]?.bytes, payload); + }); + + it("round-trips a FAT-resident stream whose size is not a whole multiple of the sector size", () => { + const payload = enc("C".repeat(5000)); + const streams = readCompoundFile( + writeCompoundFile([stream("Ragged", payload)]), + ); + expectSameBytes(streams[0]?.bytes, payload); + }); + + it("round-trips a zero-length stream", () => { + const streams = readCompoundFile( + writeCompoundFile([ + stream("Empty", new Uint8Array(0)), + stream("Other", enc("x")), + ]), + ); + expect(streams.map((s) => s.path)).toEqual(["Empty", "Other"]); + expect(streams[0]?.bytes.length).toBe(0); + }); + + it("round-trips a file holding no streams at all", () => { + const bytes = writeCompoundFile([]); + expect(readCompoundFile(bytes)).toEqual([]); + expect(u32(bytes, 0x3c)).toBe(ENDOFCHAIN); // no mini FAT is required when no stream is mini-resident + expect(u32(bytes, 512 * 2 + 0x74)).toBe(ENDOFCHAIN); // ... and the root entry's own starting sector says the same + }); + + it("round-trips the mix of mini- and FAT-resident streams the four binary-format codecs actually write", () => { + // The real shape: one large content stream plus a small companion. 'Current User' is a few dozen bytes in a genuine .ppt, so the mini path is not a corner case for these consumers -- it is the normal case for half their streams. + const document = enc("D".repeat(20000)); + const currentUser = enc("E".repeat(48)); + const table = enc("F".repeat(9000)); + const streams = readCompoundFile( + writeCompoundFile([ + stream("PowerPoint Document", document), + stream("Current User", currentUser), + stream("1Table", table), + ]), + ); + expectSameBytes( + streams.find((s) => s.path === "PowerPoint Document")?.bytes, + document, + ); + expectSameBytes( + streams.find((s) => s.path === "Current User")?.bytes, + currentUser, + ); + expectSameBytes(streams.find((s) => s.path === "1Table")?.bytes, table); + }); + + it("round-trips streams nested inside storages, at more than one level", () => { + const inner = enc("nested payload"); + const streams = readCompoundFile( + writeCompoundFile([ + stream("ObjectPool/_1234/Package", inner), + stream("ObjectPool/_1234/CompObj", enc("compobj")), + stream("WordDocument", enc("G".repeat(6000))), + ]), + ); + expect(streams.map((s) => s.path).sort()).toEqual([ + "ObjectPool/_1234/CompObj", + "ObjectPool/_1234/Package", + "WordDocument", + ]); + expectSameBytes( + streams.find((s) => s.path === "ObjectPool/_1234/Package")?.bytes, + inner, + ); + }); + + it("round-trips every stream of a file needing several directory sectors", () => { + // 40 entries at 4 per 512-byte directory sector needs 10 chained directory sectors, and 40 siblings make the red-black tree several levels deep. + const inputs = Array.from({ length: 40 }, (_unused, index) => + stream(`Stream${index}`, enc(`payload ${index}`.repeat(index + 1))), + ); + const streams = readCompoundFile(writeCompoundFile(inputs)); + expect(streams).toHaveLength(40); + for (const input of inputs) { + expectSameBytes( + streams.find((s) => s.path === input.path)?.bytes, + input.bytes, + ); + } + }); + + it("round-trips a stream large enough to need several FAT sectors", () => { + // A 512-byte sector's FAT maps 128 sectors, i.e. 64 KiB of file, so a 300 KiB stream forces the FAT itself to span several sectors and to reach its own fixed point against the total sector count. + const payload = new Uint8Array(300 * 1024); + for (let i = 0; i < payload.length; i++) { + payload[i] = (i * 31 + 7) & 0xff; + } + const bytes = writeCompoundFile([stream("Workbook", payload)]); + expect(u32(bytes, 0x2c)).toBeGreaterThan(1); + expectSameBytes(readCompoundFile(bytes)[0]?.bytes, payload); + }); + + it("round-trips a file large enough to need a DIFAT sector chain", () => { + // The header's own DIFAT array holds 109 FAT sector locations, each FAT sector mapping 128 sectors of 512 bytes: 6.875 MiB ([MS-CFB] 2.5). A stream past that forces the writer to spill into chained DIFAT sectors, which is squarely inside the size range a real .doc or .xls reaches. + const payload = new Uint8Array(8 * 1024 * 1024); + for (let i = 0; i < payload.length; i++) { + payload[i] = (i * 17 + 3) & 0xff; + } + const bytes = writeCompoundFile([stream("WordDocument", payload)]); + expect(u32(bytes, 0x2c)).toBeGreaterThan(109); // more FAT sectors than the header array can name + expect(u32(bytes, 0x48)).toBeGreaterThan(0); // ... so DIFAT sectors exist + expect(u32(bytes, 0x44)).not.toBe(ENDOFCHAIN); // ... and the header names the first of them + const streams = readCompoundFile(bytes); + expectSameBytes(streams[0]?.bytes, payload); + }); + + it("marks DIFAT sectors as DIFSECT in the FAT rather than chaining them there [MS-CFB] 2.5", () => { + const payload = new Uint8Array(8 * 1024 * 1024); + const bytes = writeCompoundFile([stream("WordDocument", payload)]); + const fatSectorCount = u32(bytes, 0x2c); + const difatSector = u32(bytes, 0x44); + // The FAT is contiguous from sector 0 in this writer's layout, so the FAT entry describing the first DIFAT sector sits in the FAT sector holding index difatSector. + const holder = u32(bytes, 0x4c + Math.floor(difatSector / 128) * 4); + expect(u32(bytes, (holder + 1) * 512 + (difatSector % 128) * 4)).toBe( + DIFSECT, + ); + expect(u32(bytes, 512 + 0 * 4)).toBe(FATSECT); + expect(fatSectorCount).toBeGreaterThan(109); + }); + + it("round-trips a version 4 file, mini and FAT paths both", () => { + const small = enc("small under the cutoff"); + const large = enc("H".repeat(20000)); + const streams = readCompoundFile( + writeCompoundFile([stream("Large", large), stream("Small", small)], { + majorVersion: 4, + }), + ); + expectSameBytes(streams.find((s) => s.path === "Small")?.bytes, small); + expectSameBytes(streams.find((s) => s.path === "Large")?.bytes, large); + }); + + it("re-writes what it read, byte-identically, so read -> write -> read is a fixed point", () => { + const original = writeCompoundFile([ + stream("Workbook", enc("I".repeat(9000))), + stream("Storage/Inner", enc("inner")), + stream("Current User", enc("J".repeat(40))), + ]); + const rewritten = writeCompoundFile(readCompoundFile(original)); + expect(firstDifference(rewritten, original)).toBe(-1); + }); + + it("emits identical bytes regardless of the order the streams are supplied in", () => { + // The directory's order is the format's own name ordering, not the caller's, so two callers building the same file from differently ordered lists must not produce different bytes. + const a = writeCompoundFile([ + stream("Zeta", enc("z")), + stream("Alpha", enc("a")), + stream("Beta", enc("b")), + ]); + const b = writeCompoundFile([ + stream("Beta", enc("b")), + stream("Zeta", enc("z")), + stream("Alpha", enc("a")), + ]); + expect(firstDifference(a, b)).toBe(-1); + }); +}); + +describe("writeCompoundFile directory red-black trees [MS-CFB] 2.6.4", () => { + it("orders siblings by name length first, then by uppercased code point", () => { + // 'Z' and 'B' are both one code point, so they compare by character; 'AA' is longer and therefore greater than both, even though 'A' < 'B' < 'Z' alphabetically. readCompoundFile walks the tree in order, so its output order is the tree's own sorted order. + const streams = readCompoundFile( + writeCompoundFile([ + stream("AA", enc("1")), + stream("Z", enc("2")), + stream("B", enc("3")), + ]), + ); + expect(streams.map((s) => s.path)).toEqual(["B", "Z", "AA"]); + }); + + it("treats names differing only in case as the same sibling, and rejects the collision", () => { + expect(() => + writeCompoundFile([stream("Table", enc("1")), stream("TABLE", enc("2"))]), + ).toThrow(CompoundFileWriteError); + }); + + it("satisfies every red-black constraint, including black height, for a large sibling set", () => { + const inputs = Array.from({ length: 63 }, (_unused, index) => + stream(`Entry${index}`, enc(`v${index}`)), + ); + const entries = parseDirectory(writeCompoundFile(inputs)); + const root = entries[0]; + expect(root?.objectType).toBe(5); + expect(root?.colour).toBe(1); + expectRedBlackTree(entries, root?.child ?? NOSTREAM); + }); + + it("satisfies the red-black constraints for every storage's own sibling set, not just the root's", () => { + const inputs = [ + ...Array.from({ length: 17 }, (_unused, index) => + stream(`Pool/Item${index}`, enc(`p${index}`)), + ), + ...Array.from({ length: 9 }, (_unused, index) => + stream(`Top${index}`, enc(`t${index}`)), + ), + ]; + const entries = parseDirectory(writeCompoundFile(inputs)); + for (const entry of entries) { + if (entry.objectType === 1 || entry.objectType === 5) { + expectRedBlackTree(entries, entry.child); + } + } + }); + + it("gives a lone sibling a black node rather than a red root", () => { + const entries = parseDirectory( + writeCompoundFile([stream("Only", enc("x"))]), + ); + expect(entries[1]?.colour).toBe(1); + expect(entries[1]?.left).toBe(NOSTREAM); + expect(entries[1]?.right).toBe(NOSTREAM); + }); + + it("writes storage entries with a zeroed starting sector and size, as [MS-CFB] 2.6.1 requires", () => { + const entries = parseDirectory( + writeCompoundFile([stream("Pool/Inner", enc("x"))]), + ); + const storage = entries.find((entry) => entry.objectType === 1); + expect(storage?.name).toBe("Pool"); + expect(storage?.startSector).toBe(0); + expect(storage?.size).toBe(0); + }); +}); + +describe("writeCompoundFile input validation", () => { + it("rejects a name holding one of the characters [MS-CFB] 2.6.1 forbids", () => { + for (const name of ["back\\slash", "colon:name", "bang!name"]) { + expect(() => writeCompoundFile([stream(name, enc("x"))])).toThrow( + CompoundFileWriteError, + ); + } + }); + + it("accepts the control-prefixed names the office binary formats genuinely use", () => { + // '\x05SummaryInformation' and '\x01CompObj' are real stream names; only '/', '\\', ':' and '!' are forbidden, so a reserved-range prefix must pass through untouched. + const name = "SummaryInformation"; + const streams = readCompoundFile( + writeCompoundFile([stream(name, enc("summary"))]), + ); + expect(streams.map((s) => s.path)).toEqual([name]); + }); + + it("rejects a name longer than the 32 code points the directory entry holds", () => { + expect(() => writeCompoundFile([stream("N".repeat(32), enc("x"))])).toThrow( + CompoundFileWriteError, + ); + expect(() => + writeCompoundFile([stream("N".repeat(31), enc("x"))]), + ).not.toThrow(); + }); + + it("rejects an empty path or an empty path segment", () => { + for (const path of ["", "/Leading", "Trailing/", "Double//Segment"]) { + expect(() => writeCompoundFile([stream(path, enc("x"))])).toThrow( + CompoundFileWriteError, + ); + } + }); + + it("rejects the same path supplied twice", () => { + expect(() => + writeCompoundFile([stream("Dup", enc("1")), stream("Dup", enc("2"))]), + ).toThrow(CompoundFileWriteError); + }); + + it("rejects a path that needs one name to be both a storage and a stream", () => { + expect(() => + writeCompoundFile([ + stream("Thing", enc("1")), + stream("Thing/Inner", enc("2")), + ]), + ).toThrow(CompoundFileWriteError); + expect(() => + writeCompoundFile([ + stream("Thing/Inner", enc("2")), + stream("Thing", enc("1")), + ]), + ).toThrow(CompoundFileWriteError); + }); + + it("names the offending path in the error it throws", () => { + expect(() => writeCompoundFile([stream("bad:name", enc("x"))])).toThrow( + /bad:name/, + ); + }); +}); diff --git a/packages/archive-codec/src/cfb/write.ts b/packages/archive-codec/src/cfb/write.ts new file mode 100644 index 00000000..6206e3cb --- /dev/null +++ b/packages/archive-codec/src/cfb/write.ts @@ -0,0 +1,521 @@ +import type { CompoundFileStream } from "./read"; + +// The write half of the classic OLE compound-file container ([MS-CFB]): given the same named-stream vocabulary readCompoundFile returns, it emits a conformant compound file -- header, FAT, DIFAT (header array and chained DIFAT sectors), directory entries as genuine red-black sibling trees, and the mini-FAT/mini-stream allocation small streams take. It exists because the family's legacy binary codecs (doc-codec, xls-codec, ppt-codec, wpd-codec) can read their [MS-CFB]-contained formats but cannot produce them: a .doc, .xls, or .ppt writer needs a compound file to put its own binary streams into, and that container is structural knowledge exactly as the reader's is -- sectors, chains, and directory entries, never that any stream is a document (see documents.js#815, #816, #817). +// +// Deliberately the mirror image of readCompoundFile: it takes the array that returns, so writeCompoundFile(readCompoundFile(bytes)) is a well-typed round trip rather than a translation between two vocabularies. Nested storages come with that symmetry -- the reader emits slash-joined paths for streams inside storages, so a writer that could not accept one would not be able to re-write what its own package had just read, even though no legacy-format codec needs nesting for its own streams. +// +// Size, and why only one ceiling is checked. The header's own DIFAT array names 109 FAT sectors, which for version 3 covers 6.875 MB ([MS-CFB] 2.5) -- a limit a real .doc or .xls passes routinely, so chained DIFAT sectors are written rather than a size cap being imposed. What remains is the version 3 per-stream ceiling of 0x80000000 bytes, which is checked and throws, because past it the 64-bit stream-size field would need its high half and the spec forbids that in a version 3 file. The format's own sector-number ceiling (MAXREGSECT, 0xFFFFFFFA) is not checked because it cannot be reached: at the smallest sector size it stands for roughly 2 TB, and this writer builds the file in one Uint8Array, whose own allocation limit is orders of magnitude lower and fails loudly on its own. + +// [MS-CFB] 2.3 special FAT values, and 2.6.1's sibling/child terminator. Restated here rather than imported from ./read: the reader keeps them private, and a writer that shared a mutable module-level surface with the reader would couple the two halves for no gain beyond four constants. +const ENDOFCHAIN = 0xfffffffe; +const FREESECT = 0xffffffff; +const FATSECT = 0xfffffffd; +const DIFSECT = 0xfffffffc; +const NOSTREAM = 0xffffffff; +// A FREESECT is four 0xFF bytes, so filling a byte range with this is filling it with FREESECT entries -- which is how every FAT, mini-FAT, and DIFAT region below starts out, and how the spec's requirement that entries past the end of the file read FREESECT is met without a second pass over the tail. +const FREESECT_FILL_BYTE = FREESECT & 0xff; +// [MS-CFB] 2.2: the header is 512 bytes whatever the sector size, and its own DIFAT array names the first 109 FAT sectors. +const HEADER_DIFAT_ENTRIES = 109; +const HEADER_DIFAT_OFFSET = 0x4c; +// [MS-CFB] 2.6.1: every directory entry is exactly 128 bytes, and its name field holds at most 32 UTF-16 code points including the terminating null. +const DIRECTORY_ENTRY_SIZE = 128; +const MAX_NAME_CODE_UNITS = 31; +// [MS-CFB] 2.2: the mini sector size is fixed at 2^6, and the cutoff MUST be written as 0x00001000 -- a stream at or above it is allocated from the FAT, below it from the mini FAT. +const MINI_SECTOR_SHIFT = 6; +const MINI_SECTOR_SIZE = 1 << MINI_SECTOR_SHIFT; +const MINI_STREAM_CUTOFF = 0x1000; +// [MS-CFB] 2.6.1 object types and colour flags. +const OBJECT_TYPE_STORAGE = 1; +const OBJECT_TYPE_STREAM = 2; +const OBJECT_TYPE_ROOT = 5; +const COLOUR_RED = 0; +const COLOUR_BLACK = 1; +// The first directory entry's name is not load-bearing -- readers reach the entry by its position and its type-5 object type -- but "Root Entry" is what every producer writes and what an inspecting human expects to see. +const ROOT_ENTRY_NAME = "Root Entry"; +// [MS-CFB] 2.6.1: a version 3 file's stream size MUST be at most 0x80000000, so the high half of the 64-bit size field is always zero there. Version 4 has no such ceiling. +const MAX_VERSION_3_STREAM_BYTES = 0x80000000; +// [MS-CFB] 2.6.1: '/' cannot reach a name at all, since this API spells storage nesting with it; the other three are rejected here. +const ILLEGAL_NAME_CHARACTERS = ["\\", ":", "!"] as const; + +// Thrown when the streams a caller asked to write cannot be expressed as a conformant compound file: an illegal or over-long name, an empty path segment, or two siblings whose names collide under the format's own case-insensitive ordering. A distinct class from the reader's CompoundFileFormatError because the two describe opposite failures -- that one says the bytes handed in are malformed, this one says the request is unwritable -- and a consumer that catches one has no business swallowing the other. +export class CompoundFileWriteError extends Error { + constructor(message: string) { + super(message); + this.name = "CompoundFileWriteError"; + } +} + +export interface WriteCompoundFileOptions { + // Version 3 (512-byte sectors) unless named otherwise: it is what every legacy Office binary format is written as, and what the four codecs consuming this writer produce. Version 4 (4096-byte sectors) writes the same structures with the header zero-padded out to its full first sector. + readonly majorVersion?: 3 | 4; +} + +interface StorageNode { + readonly name: string; + readonly children: TreeNode[]; +} + +interface StreamNode { + readonly name: string; + readonly bytes: Uint8Array; +} + +// Storage and stream are distinguished by which field each carries rather than by a tag: no node can hold both, so the presence of `children` already discriminates the union and a `kind` field would be a second source of truth for the same fact. +type TreeNode = StorageNode | StreamNode; + +function isStorage(node: TreeNode): node is StorageNode { + return "children" in node; +} + +// [MS-CFB] 2.6.1 gives the first directory entry its own object type: entry 0 is the root storage (5), every other storage an ordinary one (1), and everything else a stream (2). +function objectTypeOf(entry: PlannedEntry): number { + if (entry.id === 0) { + return OBJECT_TYPE_ROOT; + } + return isStorage(entry.node) ? OBJECT_TYPE_STORAGE : OBJECT_TYPE_STREAM; +} + +// One directory entry under construction. Its name, type, and content are fixed when the path tree is built; its sibling links and colour are filled in by the red-black construction, and its sector and size once the layout is known -- three passes over one object rather than three parallel arrays indexed by entry id. +interface PlannedEntry { + readonly id: number; + readonly node: TreeNode; + left: number; + right: number; + child: number; + colour: number; + startSector: number; + size: number; +} + +// One stream's entry paired with the bytes it carries, for the two allocation partitions. The pairing is what keeps the emission passes free of a storage case they can never actually meet: a partition is built where the union is already narrowed, so nothing downstream has to narrow it again. +interface ResidentStream { + readonly entry: PlannedEntry; + readonly bytes: Uint8Array; +} + +// [MS-CFB] 2.6.4 uppercases one UTF-16 code point at a time using the simple (single-code-point) case mapping. JavaScript's toUpperCase applies the FULL mapping, which can expand one code unit into several ('ß' becomes 'SS'); wherever it does, the simple mapping is the identity, so an expansion means the code unit is left alone. Surrogates are never uppercased, because the spec's mapping is per code point and a surrogate is half of one. +function upperCodeUnit(value: string, index: number): number { + const unit = value.charCodeAt(index); + if (unit >= 0xd800 && unit <= 0xdfff) { + return unit; + } + const upper = String.fromCharCode(unit).toUpperCase(); + return upper.length === 1 ? upper.charCodeAt(0) : unit; +} + +// The [MS-CFB] 2.6.4 sorting relationship: a shorter name is less than a longer one, and equal-length names compare by uppercased UTF-16 code point. Length is compared as the code-unit count rather than the Directory Entry Name Length field the spec names, because that field is exactly (code units + 1) * 2 -- a strictly increasing function of the same quantity, so the two orderings are identical. Names that compare equal are the same name to the format, which is why this doubles as the sibling-uniqueness test. +function compareEntryNames(left: string, right: string): number { + if (left.length !== right.length) { + return left.length - right.length; + } + for (let i = 0; i < left.length; i++) { + const difference = upperCodeUnit(left, i) - upperCodeUnit(right, i); + if (difference !== 0) { + return difference; + } + } + return 0; +} + +function checkedSegment(name: string, path: string): string { + if (name.length === 0) { + throw new CompoundFileWriteError( + `stream path ${JSON.stringify(path)} has an empty name segment; every segment must name a storage, and the last must name the stream`, + ); + } + if (name.length > MAX_NAME_CODE_UNITS) { + throw new CompoundFileWriteError( + `'${name}' is ${name.length} UTF-16 code points, more than the ${MAX_NAME_CODE_UNITS} a directory entry's name field holds alongside its terminating null (in stream path ${JSON.stringify(path)})`, + ); + } + for (const illegal of ILLEGAL_NAME_CHARACTERS) { + if (name.includes(illegal)) { + throw new CompoundFileWriteError( + `'${name}' holds '${illegal}', which [MS-CFB] 2.6.1 forbids in a storage or stream name (in stream path ${JSON.stringify(path)})`, + ); + } + } + return name; +} + +// Grafts one stream onto the storage tree, creating the storages its path names along the way. Sibling identity is the format's own ordering, not string equality, so 'Table' and 'TABLE' collide here exactly as they would for a reader searching the sibling tree. +function addStream( + root: StorageNode, + path: string, + bytes: Uint8Array, +): void { + const segments = path.split("/"); + let storage = root; + let depth = 0; + for (const segment of segments) { + depth += 1; + const name = checkedSegment(segment, path); + const existing = storage.children.find( + (child) => compareEntryNames(child.name, name) === 0, + ); + if (depth === segments.length) { + if (existing !== undefined) { + throw new CompoundFileWriteError( + `stream path ${JSON.stringify(path)} collides with '${existing.name}', which the file already holds in the same storage ([MS-CFB] 2.6.4 requires siblings to have unique names)`, + ); + } + storage.children.push({ name, bytes }); + } else if (existing === undefined) { + const created: StorageNode = { name, children: [] }; + storage.children.push(created); + storage = created; + } else if (isStorage(existing)) { + storage = existing; + } else { + throw new CompoundFileWriteError( + `stream path ${JSON.stringify(path)} needs '${existing.name}' to be a storage, but the file already holds a stream by that name`, + ); + } + } +} + +// The depth of the deepest node in the balanced tree linkSiblings builds over `count` siblings. Each recursion halves the sibling count, so the deepest node sits at floor(log2(count)) -- computed by bit length rather than Math.log2, which is a float operation whose rounding at exact powers of two would silently mis-colour a whole level. +function deepestDepth(count: number): number { + return count === 0 ? 0 : 31 - Math.clz32(count); +} + +// Builds one storage's sibling red-black tree over its already-sorted children, returning its root, and satisfies every [MS-CFB] 2.6.4 constraint by construction rather than by rebalancing: splitting a sorted list at its midpoint gives a binary search tree whose nodes sit at depths 0..D for D = floor(log2 n) and whose empty positions sit at depths no shallower than floor(log2(n+1)) >= D, so colouring exactly the depth-D nodes red makes every root-to-leaf path carry D + 1 black nodes (a path reaching depth D + 1 does so only through a red node, which adds none) with no two reds adjacent (reds share only black parents at depth D - 1) and a black root (depth 0 is red only when D is 0, the lone-sibling case, which is coloured black instead). +// +// The spec permits the degenerate all-black colouring, and readers that only traverse would accept a right-sibling chain too -- but a chain is not a search tree, so a reader that binary-searches the siblings by name (which is what the tree is for) would fail to find entries in one. Building the balanced tree costs nothing here and is the shape real producers emit. +function linkSiblings( + siblings: readonly PlannedEntry[], + depth: number, + deepest: number, +): PlannedEntry | undefined { + const midpoint = siblings.length >> 1; + const before = siblings.slice(0, midpoint); + const [node, ...after] = siblings.slice(midpoint); + if (node === undefined) { + return undefined; + } + node.colour = depth === deepest && deepest > 0 ? COLOUR_RED : COLOUR_BLACK; + const left = linkSiblings(before, depth + 1, deepest); + const right = linkSiblings(after, depth + 1, deepest); + node.left = left === undefined ? NOSTREAM : left.id; + node.right = right === undefined ? NOSTREAM : right.id; + return node; +} + +// The planned directory, with its root entry named separately: entry 0 is always the root storage, and carrying it out of the plan directly is what lets the mini stream's own location be written to it later without an indexed lookup whose absent case could not happen. +interface DirectoryPlan { + readonly rootPlan: PlannedEntry; + readonly plans: readonly PlannedEntry[]; +} + +// Assigns directory entry ids and sibling trees. Ids run breadth-first with each storage's children in the format's own name order, so the directory a caller gets back depends only on the set of paths, never on the order they were supplied in -- two callers building the same file from differently ordered lists produce identical bytes. The walk is iterative because storage nesting depth is whatever the caller's paths say it is, and a deep path must not become a deep call stack. +function planDirectory(root: StorageNode): DirectoryPlan { + const plans: PlannedEntry[] = []; + const plan = (node: TreeNode): PlannedEntry => { + const created: PlannedEntry = { + id: plans.length, + node, + left: NOSTREAM, + right: NOSTREAM, + child: NOSTREAM, + colour: COLOUR_BLACK, + startSector: 0, + size: 0, + }; + plans.push(created); + return created; + }; + + const rootPlan = plan(root); + let frontier: PlannedEntry[] = [rootPlan]; + while (frontier.length > 0) { + const next: PlannedEntry[] = []; + for (const parent of frontier) { + const node = parent.node; + if (!isStorage(node)) { + continue; + } + node.children.sort((left, right) => + compareEntryNames(left.name, right.name), + ); + const children: PlannedEntry[] = []; + for (const child of node.children) { + const childPlan = plan(child); + children.push(childPlan); + next.push(childPlan); + } + const subtree = linkSiblings(children, 0, deepestDepth(children.length)); + parent.child = subtree === undefined ? NOSTREAM : subtree.id; + } + frontier = next; + } + return { rootPlan, plans }; +} + +// Writes the streams as a compound file. Version 3 (512-byte sectors) unless options say otherwise. Throws CompoundFileWriteError when the request itself cannot be expressed -- an illegal name, an empty path segment, colliding siblings, or a version 3 stream past the 2 GB the format allows one -- rather than emitting a file that only looks valid. +export function writeCompoundFile( + streams: readonly CompoundFileStream[], + options: WriteCompoundFileOptions = {}, +): Uint8Array { + const majorVersion = options.majorVersion ?? 3; + const sectorShift = majorVersion === 4 ? 12 : 9; + const sectorSize = 1 << sectorShift; + const entriesPerFatSector = sectorSize / 4; + const entriesPerDirectorySector = sectorSize / DIRECTORY_ENTRY_SIZE; + // A DIFAT sector spends its last slot on the pointer to the next one ([MS-CFB] 2.5), so it names one fewer FAT sector than a FAT sector holds entries. + const difatEntriesPerSector = entriesPerFatSector - 1; + + const root: StorageNode = { name: ROOT_ENTRY_NAME, children: [] }; + for (const { path, bytes } of streams) { + if (majorVersion === 3 && bytes.length > MAX_VERSION_3_STREAM_BYTES) { + throw new CompoundFileWriteError( + `stream ${JSON.stringify(path)} is ${bytes.length} bytes, past the ${MAX_VERSION_3_STREAM_BYTES}-byte ceiling [MS-CFB] 2.6.1 puts on a version 3 stream; write the file as version 4 instead`, + ); + } + addStream(root, path, bytes); + } + const { rootPlan, plans } = planDirectory(root); + + // Streams split by the header's own cutoff: at or above it a stream gets whole FAT-chained sectors, below it a run of 64-byte mini sectors carved out of the root entry's own stream. A zero-length stream takes neither -- it has no chain at all, and its starting sector is meaningless ([MS-CFB] 2.6.1). Each partition carries its bytes alongside its entry, so the emission passes below never have to re-narrow a storage back out of a list that by construction holds only streams. + const miniResident: ResidentStream[] = []; + const fatResident: ResidentStream[] = []; + for (const entry of plans) { + const node = entry.node; + if (isStorage(node)) { + continue; + } + entry.size = node.bytes.length; + if (node.bytes.length === 0) { + entry.startSector = ENDOFCHAIN; + } else if (node.bytes.length < MINI_STREAM_CUTOFF) { + miniResident.push({ entry, bytes: node.bytes }); + } else { + fatResident.push({ entry, bytes: node.bytes }); + } + } + + // The mini stream: each small stream padded out to a whole number of mini sectors and concatenated, so a stream's starting mini sector is where its own run begins. + let miniSectorCount = 0; + for (const { entry, bytes } of miniResident) { + entry.startSector = miniSectorCount; + miniSectorCount += Math.ceil(bytes.length / MINI_SECTOR_SIZE); + } + const miniStreamBytes = miniSectorCount * MINI_SECTOR_SIZE; + + const directorySectorCount = Math.ceil( + plans.length / entriesPerDirectorySector, + ); + const miniStreamSectorCount = Math.ceil(miniStreamBytes / sectorSize); + const miniFatSectorCount = Math.ceil(miniSectorCount / entriesPerFatSector); + let fatStreamSectorCount = 0; + for (const { bytes } of fatResident) { + fatStreamSectorCount += Math.ceil(bytes.length / sectorSize); + } + + // The FAT has to map every sector of the file including its own, and past 109 FAT sectors the DIFAT spills out of the header into sectors that are themselves part of the file: two counts, each defined in terms of a total that both of them grow. Both are monotonically non-decreasing in that total and bounded by it, so iterating from the smallest possible pair reaches the fixed point rather than oscillating. + const totalSectorsGiven = (fat: number, difat: number): number => + fat + + difat + + directorySectorCount + + fatStreamSectorCount + + miniStreamSectorCount + + miniFatSectorCount; + let fatSectorCount = 1; + let difatSectorCount = 0; + for (;;) { + const neededFat = Math.max( + 1, + Math.ceil( + totalSectorsGiven(fatSectorCount, difatSectorCount) / + entriesPerFatSector, + ), + ); + const neededDifat = + neededFat <= HEADER_DIFAT_ENTRIES + ? 0 + : Math.ceil((neededFat - HEADER_DIFAT_ENTRIES) / difatEntriesPerSector); + if (neededFat === fatSectorCount && neededDifat === difatSectorCount) { + break; + } + fatSectorCount = neededFat; + difatSectorCount = neededDifat; + } + const totalSectors = totalSectorsGiven(fatSectorCount, difatSectorCount); + + // Sector allocation, in the order the sectors are laid out in the file. Nothing in [MS-CFB] fixes an order; putting the FAT first makes the header's DIFAT array the identity run 0..fatSectorCount-1, and grouping each kind contiguously makes every chain a run of consecutive sectors. + const difatStart = fatSectorCount; + const directoryStart = difatStart + difatSectorCount; + let nextSector = directoryStart + directorySectorCount; + for (const { entry, bytes } of fatResident) { + entry.startSector = nextSector; + nextSector += Math.ceil(bytes.length / sectorSize); + } + const miniStreamStart = nextSector; + nextSector += miniStreamSectorCount; + const miniFatStart = nextSector; + + rootPlan.startSector = miniSectorCount === 0 ? ENDOFCHAIN : miniStreamStart; + rootPlan.size = miniStreamBytes; + + // Sector N occupies bytes [(N + 1) * sectorSize, (N + 2) * sectorSize) ([MS-CFB] 2.3): the header takes the whole first sector, which for version 4 means its 512 bytes followed by 3584 zero bytes of padding the allocation already provides. + const file = new Uint8Array(sectorSize * (1 + totalSectors)); + const view = new DataView(file.buffer); + const putU16 = (offset: number, value: number): void => { + view.setUint16(offset, value, true); + }; + const putU32 = (offset: number, value: number): void => { + view.setUint32(offset, value, true); + }; + const sectorOffset = (sector: number): number => (sector + 1) * sectorSize; + + file.fill( + FREESECT_FILL_BYTE, + sectorOffset(0), + sectorOffset(0) + fatSectorCount * sectorSize, + ); + file.fill( + FREESECT_FILL_BYTE, + sectorOffset(miniFatStart), + sectorOffset(miniFatStart) + miniFatSectorCount * sectorSize, + ); + file.fill( + FREESECT_FILL_BYTE, + sectorOffset(difatStart), + sectorOffset(difatStart) + difatSectorCount * sectorSize, + ); + file.fill( + FREESECT_FILL_BYTE, + HEADER_DIFAT_OFFSET, + HEADER_DIFAT_OFFSET + HEADER_DIFAT_ENTRIES * 4, + ); + + const setFat = (sector: number, value: number): void => { + putU32( + sectorOffset(Math.floor(sector / entriesPerFatSector)) + + (sector % entriesPerFatSector) * 4, + value, + ); + }; + const chainSectors = (start: number, count: number): void => { + for (let i = 0; i < count; i++) { + setFat(start + i, i === count - 1 ? ENDOFCHAIN : start + i + 1); + } + }; + + // The FAT describes its own sectors and the DIFAT's with role markers rather than chaining them ([MS-CFB] 2.3, 2.5); everything else is a chain. + for (let i = 0; i < fatSectorCount; i++) { + setFat(i, FATSECT); + } + for (let i = 0; i < difatSectorCount; i++) { + setFat(difatStart + i, DIFSECT); + } + chainSectors(directoryStart, directorySectorCount); + for (const { entry, bytes } of fatResident) { + chainSectors(entry.startSector, Math.ceil(bytes.length / sectorSize)); + } + chainSectors(miniStreamStart, miniStreamSectorCount); + chainSectors(miniFatStart, miniFatSectorCount); + + // The DIFAT: index n names the (n+1)th FAT sector, the header carrying the first 109 and chained DIFAT sectors the rest, each spending its last slot on the next sector's location and the last of them on ENDOFCHAIN. + for (let i = 0; i < Math.min(fatSectorCount, HEADER_DIFAT_ENTRIES); i++) { + putU32(HEADER_DIFAT_OFFSET + i * 4, i); + } + for (let sector = 0; sector < difatSectorCount; sector++) { + const base = sectorOffset(difatStart + sector); + for (let i = 0; i < difatEntriesPerSector; i++) { + const fatIndex = + HEADER_DIFAT_ENTRIES + sector * difatEntriesPerSector + i; + if (fatIndex < fatSectorCount) { + putU32(base + i * 4, fatIndex); + } + } + putU32( + base + difatEntriesPerSector * 4, + sector === difatSectorCount - 1 ? ENDOFCHAIN : difatStart + sector + 1, + ); + } + + // The mini FAT chains each small stream's run of mini sectors, in the same order the runs were laid down in the mini stream. + const setMiniFat = (miniSector: number, value: number): void => { + putU32( + sectorOffset( + miniFatStart + Math.floor(miniSector / entriesPerFatSector), + ) + + (miniSector % entriesPerFatSector) * 4, + value, + ); + }; + for (const { entry, bytes } of miniResident) { + const count = Math.ceil(bytes.length / MINI_SECTOR_SIZE); + for (let i = 0; i < count; i++) { + setMiniFat( + entry.startSector + i, + i === count - 1 ? ENDOFCHAIN : entry.startSector + i + 1, + ); + } + } + + // Stream content. Both arms write into an allocation that is already zero, so the tail of a stream's last sector (or last mini sector) is padding without a second write. + for (const { entry, bytes } of fatResident) { + file.set(bytes, sectorOffset(entry.startSector)); + } + for (const { entry, bytes } of miniResident) { + file.set( + bytes, + sectorOffset(miniStreamStart) + entry.startSector * MINI_SECTOR_SIZE, + ); + } + + const entryOffset = (id: number): number => + sectorOffset(directoryStart + Math.floor(id / entriesPerDirectorySector)) + + (id % entriesPerDirectorySector) * DIRECTORY_ENTRY_SIZE; + + for (const entry of plans) { + const base = entryOffset(entry.id); + const node = entry.node; + const name = node.name; + for (let i = 0; i < name.length; i++) { + putU16(base + i * 2, name.charCodeAt(i)); + } + // The already-zero code unit past the name is the terminating null the length counts. + putU16(base + 0x40, (name.length + 1) * 2); + view.setUint8(base + 0x42, objectTypeOf(entry)); + view.setUint8(base + 0x43, entry.colour); + putU32(base + 0x44, entry.left); + putU32(base + 0x48, entry.right); + putU32(base + 0x4c, entry.child); + // CLSID (0x50), state bits (0x60), creation time (0x64), and modified time (0x6c) stay zero: [MS-CFB] 2.6.1 requires that of a stream entry and of the root's timestamps, and an implementation that does not let callers set a storage's class or state bits MUST default them to zero -- which is exactly this one, since none of it survives a round trip through the stream vocabulary this writer takes. + putU32(base + 0x74, entry.startSector); + putU32(base + 0x78, entry.size >>> 0); + putU32(base + 0x7c, Math.floor(entry.size / 4294967296)); + } + // Directory entries past the last real one pad their sector out. They stay object type 0 (unallocated) with a zero-length name, and only their links need writing, since NOSTREAM is not the zero the allocation already holds. + for ( + let id = plans.length; + id < directorySectorCount * entriesPerDirectorySector; + id++ + ) { + const base = entryOffset(id); + putU32(base + 0x44, NOSTREAM); + putU32(base + 0x48, NOSTREAM); + putU32(base + 0x4c, NOSTREAM); + } + + // The header ([MS-CFB] 2.2). Header CLSID (0x08), reserved (0x22), and the transaction signature number (0x34) stay zero, each because the spec requires it. + file.set([0xd0, 0xcf, 0x11, 0xe0, 0xa1, 0xb1, 0x1a, 0xe1], 0); + putU16(0x18, 0x003e); // minor version: the value the spec names for major version 3 and 4 alike + putU16(0x1a, majorVersion); + putU16(0x1c, 0xfffe); // byte order mark: little-endian + putU16(0x1e, sectorShift); + putU16(0x20, MINI_SECTOR_SHIFT); + // The directory-sector count MUST be zero in a version 3 file -- the field is unsupported there -- and carries the real count in version 4. + putU32(0x28, majorVersion === 3 ? 0 : directorySectorCount); + putU32(0x2c, fatSectorCount); + putU32(0x30, directoryStart); + putU32(0x38, MINI_STREAM_CUTOFF); + putU32(0x3c, miniFatSectorCount === 0 ? ENDOFCHAIN : miniFatStart); + putU32(0x40, miniFatSectorCount); + putU32(0x44, difatSectorCount === 0 ? ENDOFCHAIN : difatStart); + putU32(0x48, difatSectorCount); + + return file; +} diff --git a/packages/archive-codec/src/index.test.ts b/packages/archive-codec/src/index.test.ts index 9f797714..9d1df417 100644 --- a/packages/archive-codec/src/index.test.ts +++ b/packages/archive-codec/src/index.test.ts @@ -2,8 +2,11 @@ import { describe, expect, it } from "vitest"; import { ArchiveWalkLimitError, detectArchiveFormat, + isCompoundFile, isZipArchive, + readCompoundFile, unzipPackage, + writeCompoundFile, zipPackage, walkArchive, } from "./index"; @@ -27,6 +30,19 @@ describe("archive-codec barrel smoke", () => { expect(unzipPackage(nested)["inner.txt"]).toEqual(enc.encode("inner")); }); + it("exposes both directions of the compound-file surface", () => { + const enc = new TextEncoder(); + const payload = enc.encode("a compound-file stream"); + const bytes = writeCompoundFile([ + { path: "Storage/Package", bytes: payload }, + ]); + expect(isCompoundFile(bytes)).toBe(true); + expect(detectArchiveFormat(bytes)).toBe("cfb"); + const streams = readCompoundFile(bytes); + expect(streams.map((s) => s.path)).toEqual(["Storage/Package"]); + expect(streams[0]?.bytes).toEqual(payload); + }); + it("surfaces ArchiveWalkLimitError for out-of-contract input", () => { const bytes = zipPackage([["a.bin", { bytes: new Uint8Array(64) }]]); expect(() => walkArchive(bytes, { maxTotalBytes: 8 })).toThrow( diff --git a/packages/archive-codec/src/index.ts b/packages/archive-codec/src/index.ts index c3993124..6892d003 100644 --- a/packages/archive-codec/src/index.ts +++ b/packages/archive-codec/src/index.ts @@ -1,7 +1,8 @@ -// The archive and container utility package for the documents.js family: ZIP container read/write, archive-format detection, recursive ZIP-in-ZIP walking under explicit depth and cumulative decompressed-size guards, and classic OLE compound-file (CFB) reading including the OLE Package stream wrapper an embed's real file rides in -- with zero document-format knowledge (it knows bytes and container structure, never that any entry or stream is a document). Motivated by documents.js#564: OOXML embedded-object packages are genuinely separate ZIP blobs inside the outer ZIP, and nothing in the family recursed into them safely before this. +// The archive and container utility package for the documents.js family: ZIP container read/write, archive-format detection, recursive ZIP-in-ZIP walking under explicit depth and cumulative decompressed-size guards, and classic OLE compound-file (CFB) reading and writing including the OLE Package stream wrapper an embed's real file rides in -- with zero document-format knowledge (it knows bytes and container structure, never that any entry or stream is a document). Motivated by documents.js#564: OOXML embedded-object packages are genuinely separate ZIP blobs inside the outer ZIP, and nothing in the family recursed into them safely before this. export * from "./cfb/detect"; export * from "./cfb/ole-package"; export * from "./cfb/read"; +export * from "./cfb/write"; export * from "./zip/container"; export * from "./zip/detect"; export * from "./zip/walk"; diff --git a/packages/archive-codec/test/smoke.test.mjs b/packages/archive-codec/test/smoke.test.mjs index 852b8825..a12cf124 100644 --- a/packages/archive-codec/test/smoke.test.mjs +++ b/packages/archive-codec/test/smoke.test.mjs @@ -15,10 +15,16 @@ const BARREL_FUNCTIONS = [ 'walkArchive', 'isCompoundFile', 'readCompoundFile', + 'writeCompoundFile', 'readOlePackage', ]; const BARREL_CONSTANTS = ['MAX_WALK_DEPTH', 'MAX_WALK_TOTAL_BYTES', 'MAX_CFB_TOTAL_STREAM_BYTES']; -const BARREL_CLASSES = ['ArchiveWalkLimitError', 'CompoundFileFormatError', 'OlePackageFormatError']; +const BARREL_CLASSES = [ + 'ArchiveWalkLimitError', + 'CompoundFileFormatError', + 'CompoundFileWriteError', + 'OlePackageFormatError', +]; describe('dist/ barrel exports are present in both builds', () => { for (const name of BARREL_FUNCTIONS) { @@ -51,6 +57,7 @@ describe('dist/ deep imports resolve for every advertised module, in both builds { path: '../dist/zip/walk.js', exports: ['walkArchive', 'MAX_WALK_DEPTH'] }, { path: '../dist/cfb/detect.js', exports: ['isCompoundFile'] }, { path: '../dist/cfb/read.js', exports: ['readCompoundFile', 'MAX_CFB_TOTAL_STREAM_BYTES'] }, + { path: '../dist/cfb/write.js', exports: ['writeCompoundFile', 'CompoundFileWriteError'] }, { path: '../dist/cfb/ole-package.js', exports: ['readOlePackage'] }, { path: '../dist/magic.js', exports: [] }, ]; @@ -86,4 +93,25 @@ describe('dist/ end-to-end: both builds round-trip a real archive', () => { expect(cjsEntries[0]?.bytes).toEqual(content); expect(cjs.detectArchiveFormat(zipBytes)).toBe('zip'); }); + + it('writeCompoundFile -> isCompoundFile/detectArchiveFormat -> readCompoundFile agrees across ESM and CJS', () => { + // The compound-file half of the same end-to-end check, and the one that needs both directions built: a writer whose output only its own build can read would pass every deep-import check above and still be broken. + const small = new TextEncoder().encode('smoke stream for archive-codec'); + const large = new Uint8Array(5000).fill(0x41); + const built = esm.writeCompoundFile([ + { path: 'Storage/Small', bytes: small }, + { path: 'Large', bytes: large }, + ]); + expect(esm.isCompoundFile(built)).toBe(true); + expect(esm.detectArchiveFormat(built)).toBe('cfb'); + expect(esm.isZipArchive(built)).toBe(false); + + const esmStreams = esm.readCompoundFile(built); + expect(esmStreams.map((entry) => entry.path).sort()).toEqual(['Large', 'Storage/Small']); + expect(esmStreams.find((entry) => entry.path === 'Storage/Small')?.bytes).toEqual(small); + + const cjsStreams = cjs.readCompoundFile(cjs.writeCompoundFile([['Large', large]].map(([path, bytes]) => ({ path, bytes })))); + expect(cjsStreams.map((entry) => entry.path)).toEqual(['Large']); + expect(cjsStreams[0]?.bytes).toEqual(large); + }); }); diff --git a/packages/archive-codec/test/workers/archive-codec.test.ts b/packages/archive-codec/test/workers/archive-codec.test.ts index bb032a56..b1368624 100644 --- a/packages/archive-codec/test/workers/archive-codec.test.ts +++ b/packages/archive-codec/test/workers/archive-codec.test.ts @@ -2,12 +2,14 @@ import { describe, expect, it } from 'vitest'; import { ArchiveWalkLimitError, CompoundFileFormatError, + CompoundFileWriteError, detectArchiveFormat, isCompoundFile, isZipArchive, readCompoundFile, readOlePackage, unzipPackage, + writeCompoundFile, zipPackage, walkArchive, } from '../../src'; @@ -81,10 +83,31 @@ describe('archive-codec under the Cloudflare Workers runtime', () => { expect(unwrapped.fileBytes).toEqual(fileBytes); }); + it('writes a compound file and reads it back inside the isolate', () => { + // The writer is the same kind of pure integer-and-DataView work over one allocation the reader is -- one Uint8Array, one DataView, no Buffer and no fs -- so it belongs to this package's Worker-isomorphic half too. Both allocation paths run here: 'Current User' is under the 4096-byte cutoff so it lands in the mini stream, 'PowerPoint Document' above it so it takes FAT-chained sectors, and a nested storage covers the directory tree's second level. + const currentUser = encode('a small stream, mini-FAT resident'); + const document = new Uint8Array(9000).fill(0x50); + const built = writeCompoundFile([ + { path: 'PowerPoint Document', bytes: document }, + { path: 'Current User', bytes: currentUser }, + { path: 'ObjectPool/Package', bytes: encode('nested') }, + ]); + expect(isCompoundFile(built)).toBe(true); + expect(detectArchiveFormat(built)).toBe('cfb'); + const streams = readCompoundFile(built); + expect(streams.map((s) => s.path).sort()).toEqual(['Current User', 'ObjectPool/Package', 'PowerPoint Document']); + expect(streams.find((s) => s.path === 'Current User')?.bytes).toEqual(currentUser); + expect(streams.find((s) => s.path === 'PowerPoint Document')?.bytes).toEqual(document); + }); + + it('throws the named write error inside the isolate for a request that cannot be written', () => { + expect(() => writeCompoundFile([{ path: 'bad:name', bytes: encode('x') }])).toThrow(CompoundFileWriteError); + }); + it('throws the named compound-file error inside the isolate for corrupt structure', () => { const bytes = compoundFile([{ path: 'A', bytes: encode('x'.repeat(4500)) }]); const view = new DataView(bytes.buffer); view.setUint32(512 + 2 * 4, 2, true); // the stream's first data sector points at itself: a cyclic FAT chain - expect(() => readCompoundFile(bytes)).toThrowError(CompoundFileFormatError); + expect(() => readCompoundFile(bytes)).toThrow(CompoundFileFormatError); }); }); diff --git a/packages/xls-codec/README.md b/packages/xls-codec/README.md index 0676f786..a4179226 100644 --- a/packages/xls-codec/README.md +++ b/packages/xls-codec/README.md @@ -19,7 +19,7 @@ Under active development, **read-only**. Built and shipped: ### Not built yet -**The write path**, which is the substantial one. A conformant workbook has to emit a `BOF` history block, a complete `XF`/`Font`/`Format` table with the fifteen mandatory style records preceding any cell format, an `Index`/`DBCell` row-block lookup structure whose file offsets must agree with where the records actually land, and a compound-file _writer_ (`archive-codec` reads [MS-CFB] but does not write it). Shipping a plausible-looking writer that produced files Excel rejects would be worse than shipping none. Tracked on [#815](https://github.com/ExaDev/documents.js/issues/815). +**The write path**, which is the substantial one. A conformant workbook has to emit a `BOF` history block, a complete `XF`/`Font`/`Format` table with the fifteen mandatory style records preceding any cell format, an `Index`/`DBCell` row-block lookup structure whose file offsets must agree with where the records actually land, and a compound file to put the resulting `Workbook` stream into. The container half is no longer missing — [`archive-codec`](../archive-codec/README.md)'s `writeCompoundFile` writes [MS-CFB] as well as reading it — so what remains here is the BIFF8 record emission itself. Shipping a plausible-looking writer that produced files Excel rejects would be worse than shipping none. Tracked on [#815](https://github.com/ExaDev/documents.js/issues/815). Read-side gaps, each deliberate rather than overlooked: