diff --git a/README.md b/README.md index 400f804c7..f82023a9f 100644 --- a/README.md +++ b/README.md @@ -45,9 +45,9 @@ Each converts one document format to and from the shared schema, built on `docum ### Conversion engine -| Package | What it is | -| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`documents.js`](packages/documents.js/README.md) | Bidirectional docx/pptx to and from PDF conversion, and a read+write editable OOXML document model, built on `ooxml.js` and depending on every codec above that has a write path. The read-only ones (`wpd-codec`, `doc-codec`, `xls-codec`, `ppt-codec`) are not among them: a conversion-engine target needs both directions. | +| Package | What it is | +| ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [`documents.js`](packages/documents.js/README.md) | Bidirectional docx/pptx to and from PDF conversion, and a read+write editable OOXML document model, built on `ooxml.js` and depending on every write-capable codec above plus `wpd-codec` as a read-only conversion source (routable everywhere but never a target, since wpd-codec ships no writer). `doc-codec`/`xls-codec`/`ppt-codec` remain unwired: they are read-only too, but nothing yet routes them the way `wpd-codec` now is. | ### Interfaces diff --git a/packages/document-cli/README.md b/packages/document-cli/README.md index a905cd59c..266abe176 100644 --- a/packages/document-cli/README.md +++ b/packages/document-cli/README.md @@ -2,7 +2,7 @@ [![GitHub](https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white)](https://github.com/ExaDev/documents.js/tree/main/packages/document-cli) [![npm](https://img.shields.io/badge/npm-CB3837?logo=npm&logoColor=white)](https://www.npmjs.com/package/document-cli) [![npm version](https://img.shields.io/npm/v/document-cli)](https://www.npmjs.com/package/document-cli) [![CI](https://img.shields.io/github/actions/workflow/status/ExaDev/documents.js/ci.yml?branch=main)](https://github.com/ExaDev/documents.js/actions) -> A command-line interface and an interactive terminal (Ink) app for [`documents.js`](https://github.com/ExaDev/documents.js): every docx/pptx/odt/odp/ods/odg/odf/pdf/odm/odb/xlsx/csv/svg/markdown/rtf conversion, bridge, and editor documents.js exposes, plus the outline projection [`document-outline.js`](../document-outline.js/README.md) builds over any readable document, wired up as a scriptable subcommand or a full-screen terminal editor. Installs as either `document-cli` or `doculi`. +> A command-line interface and an interactive terminal (Ink) app for [`documents.js`](https://github.com/ExaDev/documents.js): every docx/pptx/odt/odp/ods/odg/odf/pdf/odm/odb/xlsx/csv/svg/markdown/rtf/wpd conversion, bridge, and editor documents.js exposes, plus the outline projection [`document-outline.js`](../document-outline.js/README.md) builds over any readable document, wired up as a scriptable subcommand or a full-screen terminal editor. Installs as either `document-cli` or `doculi`. `document-cli` adds no conversion or editing logic of its own — it is a dispatch layer over `documents.js`'s existing conversion functions, `DocumentConverter` port, live-view editors, and `.odb`/PDF readers. What it adds is two ways to drive them without writing TypeScript: a scriptable, Unix-shaped CLI (stdin/stdout, exit codes, `--json` diagnostics) for pipelines, and a full-screen Ink terminal app for browsing and editing a document interactively. @@ -143,7 +143,7 @@ document-cli pdf-inspect report.pdf document-cli pdf-inspect report.pdf --json ``` -**`fonts `** — lists every source-embedded font face a docx/pptx/odt/odp/ods/odg document carries (family, weight/style, byte length) — the same embedded faces every `-to-pdf` conversion already extracts and renders through automatically (see [Real fonts](#real-fonts) below); this command just reports what's there without converting anything. Rejects a format with no source-embedded-font concept at all (xlsx, csv, svg, pdf, markdown, odf, rtf), naming it: +**`fonts `** — lists every source-embedded font face a docx/pptx/odt/odp/ods/odg document carries (family, weight/style, byte length) — the same embedded faces every `-to-pdf` conversion already extracts and renders through automatically (see [Real fonts](#real-fonts) below); this command just reports what's there without converting anything. Rejects a format with no source-embedded-font concept at all (xlsx, csv, svg, pdf, markdown, odf, rtf, wpd), naming it: ```sh document-cli fonts report.docx @@ -155,7 +155,7 @@ document-cli fonts report.docx document-cli docx-extras report.docx ``` -**`metadata `** — prints a document's own title/author/subject/keywords/creator/producer/created/modified metadata, for any of the thirteen supported formats (docx, pptx, xlsx, odt, odp, ods, odg, svg, odf, csv, markdown, rtf, pdf); csv and svg carry no metadata container of their own, so they always report none. `--json` emits the raw metadata object: +**`metadata `** — prints a document's own title/author/subject/keywords/creator/producer/created/modified metadata, for any of the fourteen supported formats (docx, pptx, xlsx, odt, odp, ods, odg, svg, odf, csv, markdown, rtf, wpd, pdf); csv and svg carry no metadata container of their own, so they always report none. `--json` emits the raw metadata object: ```sh document-cli metadata report.pdf @@ -168,7 +168,7 @@ document-cli set-metadata report.docx report.docx --set-title "Q3 Report" --set- document-cli set-metadata report.docx report.odt --set-keywords "draft,internal" ``` -**`outline `** — prints a document's outline: the table-of-contents projection over the tree-form `DocumentTree` read straight off the source document's own bytes — headings nested by heading level, list items nested under their heading or slide, one group per slide (labelled `Slide N`), sheet (labelled with the sheet's own name), or draw page (labelled `Page N`) — rendered as indented text, two spaces per nesting depth. Leaves render their own text (a paragraph's runs, a table's cell text, an image's alt text, a formula's LaTeX) or their kind in brackets when they carry none (`[page-break]`, `[vector]`, `[embeddedObject]`). Works on any of the twelve readable formats: the command reads the source's own native tree directly (documents.js's `readNativeDocumentTree`) — no bridging conversion runs and no output bytes are discarded — and projects that tree through [`document-outline.js`](../document-outline.js/README.md)'s own `buildOutline`, this command being that package's first real consumer. `--json` emits the outline tree itself — groups as `{ text, level, children }`, leaves as the package leaves they are — rather than a CLI-private shape; a `pdf` source's own `readPdf` parse diagnostics still reach stderr exactly as they would on the matching `pdf-to-docx` command. `--from ` names the source format when the input path carries no recognised extension to infer it from — the only way to outline a document read from stdin (`-`), which otherwise has no extension to read at all. Heading nesting depends on the source document actually carrying a heading-level signal on disk (`w:outlineLvl` for docx, `text:outline-level` for odt) — present in anything authored by Word or LibreOffice, and always present for a markdown source, whose own reader parses `#`/`##` headings directly; a docx or odt this ecosystem's own writers produce does not currently stamp that signal even though the paragraph still carries its `Heading1`/`Heading2`-style reference, so outlining a docx/odt generated by this CLI's own conversions loses heading nesting specifically (list nesting and slide/sheet/page grouping are unaffected): +**`outline `** — prints a document's outline: the table-of-contents projection over the tree-form `DocumentTree` read straight off the source document's own bytes — headings nested by heading level, list items nested under their heading or slide, one group per slide (labelled `Slide N`), sheet (labelled with the sheet's own name), or draw page (labelled `Page N`) — rendered as indented text, two spaces per nesting depth. Leaves render their own text (a paragraph's runs, a table's cell text, an image's alt text, a formula's LaTeX) or their kind in brackets when they carry none (`[page-break]`, `[vector]`, `[embeddedObject]`). Works on any of the fourteen readable formats: the command reads the source's own native tree directly (documents.js's `readNativeDocumentTree`) — no bridging conversion runs and no output bytes are discarded — and projects that tree through [`document-outline.js`](../document-outline.js/README.md)'s own `buildOutline`, this command being that package's first real consumer. `--json` emits the outline tree itself — groups as `{ text, level, children }`, leaves as the package leaves they are — rather than a CLI-private shape; a `pdf` source's own `readPdf` parse diagnostics still reach stderr exactly as they would on the matching `pdf-to-docx` command. `--from ` names the source format when the input path carries no recognised extension to infer it from — the only way to outline a document read from stdin (`-`), which otherwise has no extension to read at all. Heading nesting depends on the source document actually carrying a heading-level signal on disk (`w:outlineLvl` for docx, `text:outline-level` for odt) — present in anything authored by Word or LibreOffice, and always present for a markdown source, whose own reader parses `#`/`##` headings directly; a docx or odt this ecosystem's own writers produce does not currently stamp that signal even though the paragraph still carries its `Heading1`/`Heading2`-style reference, so outlining a docx/odt generated by this CLI's own conversions loses heading nesting specifically (list nesting and slide/sheet/page grouping are unaffected): ```sh document-cli outline report.docx @@ -231,7 +231,7 @@ It supports the same seven formats documents.js's live-view editors cover — do Markdown (`.md`/`.markdown`) shares the same paragraph/run/table body-list screens docx and odt already use, through documents.js's own `MarkdownEditor` (`openMarkdown`/`createMarkdownEditor`) — a genuine live view over a mutable `ContentDocument`, the same live-view contract every other editor here follows, even though there is no `XmlElement` tree underneath it the way there is for docx/odt (`MarkdownEditor.toMarkdownText()` re-serialises the whole document fresh on every call, rather than exposing a `toBytes()`). Appending a paragraph, appending a run, and toggling bold/italic all go through the identical reducer actions docx/odt use; a markdown run has no underline, colour, font family, or font size at all (CommonMark/GFM has no construct for any of the four), so those keys — along with image insertion, which `MarkdownParagraph` has no counterpart for — are simply absent from a markdown paragraph's own key hints rather than opening a prompt that could only end in a warning. A markdown table can be created and its cells edited through the same 'T' wizard and table-view screens docx/odt use, but GFM tables have no cell-merge concept, so a merge requested alongside table creation still creates the table (unmerged) and reports why the merge itself didn't happen. `:view-source` (markdown documents only) shows the literal text the document was opened with side by side with what a save would write right now — these can genuinely differ even with no edits made this session, from a heading-style, bullet-marker, or line-ending choice the writer normalises. Every save re-serialises the whole document fresh through `buildMarkdownText`, a deliberate, permanent consequence of structured editing rather than something to work around. Diagnostics from the read side (a clamped heading level, a dropped front-matter key, a fenced code block's own info string with nowhere to go, …) now surface into the same diagnostics panel a PDF export's own substitutions already populate, the moment a `.md` file is opened, not only on export. documents.js's own `createMarkdownEditor()` exists now, but this TUI does not yet wire a "new markdown document" flow into `:new`/the new-document picker, so a markdown document can still only be opened from an existing file. -Three further kinds of format open read-only: a `.odb` browses its tables and rows with no write path at all (documents.js's own `.odb` support has no write direction to offer), a `.pdf` browses its pages and positioned items rather than being edited in place, and a `.xlsx`, `.csv`, `.svg`, or `.rtf` opens as a converted PDF preview — documents.js has no spreadsheet, svg, or rtf editor to hold a live view into, so opening one runs `xlsxToPdf`, `csvToPdf`, `svgToPdf`, or `rtfToPdf` once at open time and browses the result through the identical page-list/page-items/item-detail screens a real `.pdf` uses, with the original bytes kept alongside so a later export re-runs the same conversion with the caller's own fonts and diagnostics rather than reusing the fixed preview conversion. A `.odb` additionally browses its _structure_ alongside its data: `f` from the table list opens the form browser and `r` the report browser, each listing what the database declares and opening one to show it in full — a form's own data source and field-bound controls (sub-forms nested under their parent), a report's data-source command, band and group structure, and every `rpt:` formula. Both are rendered through the same `src/odb-structure.ts` the `odb-forms`/`odb-reports` commands print, so the two views cannot drift apart, and search filters by line (`/SUM` narrows a long report to its aggregate expressions). `Enter` on a report's own detail screen renders it — its data-source command resolved, its `rpt:` formulas evaluated, its bands laid out — to a real `docx`/`odt`/`pdf` file, through the same two-field destination-path-then-font-list form the PDF-export screen below uses; the TUI counterpart to the `odb-render-report` command. A standalone `.odf` formula document has no TUI editor either — nothing to edit interactively, only a PDF conversion. +Three further kinds of format open read-only: a `.odb` browses its tables and rows with no write path at all (documents.js's own `.odb` support has no write direction to offer), a `.pdf` browses its pages and positioned items rather than being edited in place, and a `.xlsx`, `.csv`, `.svg`, `.rtf`, or `.wpd` opens as a converted PDF preview — documents.js has no spreadsheet, svg, rtf, or wpd editor to hold a live view into (wpd-codec ships no writer at all, so a wpd document could never gain one), so opening one runs `xlsxToPdf`, `csvToPdf`, `svgToPdf`, `rtfToPdf`, or `convertDocument("wpd", "pdf", ...)` once at open time and browses the result through the identical page-list/page-items/item-detail screens a real `.pdf` uses, with the original bytes kept alongside so a later export re-runs the same conversion with the caller's own fonts and diagnostics rather than reusing the fixed preview conversion. A `.odb` additionally browses its _structure_ alongside its data: `f` from the table list opens the form browser and `r` the report browser, each listing what the database declares and opening one to show it in full — a form's own data source and field-bound controls (sub-forms nested under their parent), a report's data-source command, band and group structure, and every `rpt:` formula. Both are rendered through the same `src/odb-structure.ts` the `odb-forms`/`odb-reports` commands print, so the two views cannot drift apart, and search filters by line (`/SUM` narrows a long report to its aggregate expressions). `Enter` on a report's own detail screen renders it — its data-source command resolved, its `rpt:` formulas evaluated, its bands laid out — to a real `docx`/`odt`/`pdf` file, through the same two-field destination-path-then-font-list form the PDF-export screen below uses; the TUI counterpart to the `odb-render-report` command. A standalone `.odf` formula document has no TUI editor either — nothing to edit interactively, only a PDF conversion. The export-to-PDF screen (`e` from any editor screen) is a two-field form: a destination path, then an optional comma-separated list of local `.ttf`/`.otf` paths, which are the same `--font-file` faces the CLI takes and are derived the same way — each font's family, weight, and slope come from the file itself. `Enter` on the path field moves to the fonts field and `Enter` there exports, so leaving fonts empty is still "type a path, press Enter twice". Comma-separated rather than space-separated because a font path routinely contains spaces and almost never a comma. A face falling back to a substitute is reported into the same diagnostics panel a character substitution already is, and a bad font path fails the export with the file named, before anything is written to the destination. diff --git a/packages/document-cli/src/format.ts b/packages/document-cli/src/format.ts index 008455404..cbb09cb18 100644 --- a/packages/document-cli/src/format.ts +++ b/packages/document-cli/src/format.ts @@ -26,6 +26,7 @@ const EXTENSION_TO_FORMAT: Readonly> = { markdown: "markdown", md: "markdown", rtf: "rtf", + wpd: "wpd", pdf: "pdf", }; @@ -42,6 +43,7 @@ const FORMAT_TO_EXTENSION: Readonly> = { svg: "svg", markdown: "md", rtf: "rtf", + wpd: "wpd", pdf: "pdf", }; diff --git a/packages/document-cli/src/tui/format/export-pdf.ts b/packages/document-cli/src/tui/format/export-pdf.ts index 7316a27a2..57ad15d1e 100644 --- a/packages/document-cli/src/tui/format/export-pdf.ts +++ b/packages/document-cli/src/tui/format/export-pdf.ts @@ -1,5 +1,6 @@ import { writeFile } from "node:fs/promises"; import { + convertDocument, csvToPdf, docxToPdf, encodeMarkdownText, @@ -76,7 +77,7 @@ export async function exportToPdf( await writeFile(destinationPath, pdfBytes); return; } - // xlsx has no editor to read current bytes from (see state/types.ts's own XlsxOpenDocument doc comment) -- the original bytes captured at open time are re-converted here, with this call's own real fonts/diagnostics options, rather than reusing the fixed preview conversion `openDocumentAtPath` computed to build the read-only viewer. csv, svg, and rtf are the identical no-editor story (their own OpenDocument doc comments), each re-converted through its own to-Pdf function the same way. + // xlsx has no editor to read current bytes from (see state/types.ts's own XlsxOpenDocument doc comment) -- the original bytes captured at open time are re-converted here, with this call's own real fonts/diagnostics options, rather than reusing the fixed preview conversion `openDocumentAtPath` computed to build the read-only viewer. csv, svg, rtf, and wpd are the identical no-editor story (their own OpenDocument doc comments), each re-converted through its own to-Pdf function the same way. if (openDocument.format === "xlsx") { const pdfBytes = xlsxToPdf(openDocument.bytes, pdfOptions); await writeFile(destinationPath, pdfBytes); @@ -97,6 +98,17 @@ export async function exportToPdf( await writeFile(destinationPath, pdfBytes); return; } + // wpd has no rtfToPdf-equivalent named function (see WpdOpenDocument's own doc comment on why) -- convertDocument("wpd", "pdf", ...) reaches the identical composition-engine edge convertDocument's own named forwarders are thin wrappers over. + if (openDocument.format === "wpd") { + const pdfBytes = convertDocument( + "wpd", + "pdf", + openDocument.bytes, + pdfOptions, + ); + await writeFile(destinationPath, pdfBytes); + return; + } const bytes = openDocument.editor.toBytes(); const pdfBytes = convert(openDocument.format, bytes, pdfOptions); await writeFile(destinationPath, pdfBytes); diff --git a/packages/document-cli/src/tui/format/open-document.ts b/packages/document-cli/src/tui/format/open-document.ts index dd2e6b35b..3728e0783 100644 --- a/packages/document-cli/src/tui/format/open-document.ts +++ b/packages/document-cli/src/tui/format/open-document.ts @@ -1,5 +1,6 @@ import { readFile, writeFile } from "node:fs/promises"; import { + convertDocument, createDocx, createOdg, createOdp, @@ -110,6 +111,14 @@ export async function openDocumentAtPath( // rtf mirrors xlsx/csv/svg exactly: no live-view editor and no readRtfContent-shaped content reader wired into this TUI's editor screens, but a real rtfToPdf conversion -- opened read-only through the identical to-Pdf-then-readPdf shape. case "rtf": return { format, layout: readPdf(rtfToPdf(bytes)), bytes, path }; + // wpd mirrors rtf, but through convertDocument rather than a named wpdToPdf -- see WpdOpenDocument's own doc comment on why there is no named function for this one edge. + case "wpd": + return { + format, + layout: readPdf(convertDocument("wpd", "pdf", bytes)), + bytes, + path, + }; case "odf": throw new Error( "A standalone .odf formula document has no editor; convert it to PDF (odfToPdf) instead", @@ -152,7 +161,8 @@ export async function saveDocumentTo( openDocument.format === "xlsx" || openDocument.format === "csv" || openDocument.format === "svg" || - openDocument.format === "rtf" + openDocument.format === "rtf" || + openDocument.format === "wpd" ) { throw new Error( `A ${openDocument.format} document is opened read-only and cannot be written back`, diff --git a/packages/document-cli/src/tui/format/read-metadata.ts b/packages/document-cli/src/tui/format/read-metadata.ts index 349075731..5aaf321b0 100644 --- a/packages/document-cli/src/tui/format/read-metadata.ts +++ b/packages/document-cli/src/tui/format/read-metadata.ts @@ -10,7 +10,7 @@ import { } from "documents.js"; import type { OpenDocument } from "../state/types.js"; -// The single place in the TUI that turns an OpenDocument into its own LayoutMetadata, mirroring export-pdf.ts's own per-format dispatch style. docx/pptx/odt/odp/ods/odg all read `doc.editor.toPackage()` -- the live view's own already-decoded package (ooxml.js's for docx/pptx, odf.js's for odt/odp/ods/odg; see state/types.ts's own RULE at the top of the file for why this is called fresh on every render rather than cached), fed straight into that format's own readXContent. markdown has a live-view MarkdownEditor but no package at all -- `readMarkdownContent` runs on `doc.editor.toMarkdownText()` (re-serialised fresh, reflecting in-progress edits, matching export-pdf.ts's own convention), the same role `.editor.toPackage()` plays elsewhere. pdf, xlsx, csv, svg, and rtf all already carry a LayoutMetadata directly on `.layout` (each read-only preview format's own `.layout` is the throwaway to-Pdf-then-readPdf conversion open-document.ts already computed at open time -- see that module's own XlsxOpenDocument/CsvOpenDocument/SvgOpenDocument/RtfOpenDocument doc comments), so none of them needs a read call here at all. odb has no document-level metadata concept anywhere in this codebase (it is a table/form/report container, not a single document with its own title/author/etc.) -- this throws rather than fabricating an empty LayoutMetadata, and the metadata screen itself is the one place that catches it and shows a plain message instead of crashing. +// The single place in the TUI that turns an OpenDocument into its own LayoutMetadata, mirroring export-pdf.ts's own per-format dispatch style. docx/pptx/odt/odp/ods/odg all read `doc.editor.toPackage()` -- the live view's own already-decoded package (ooxml.js's for docx/pptx, odf.js's for odt/odp/ods/odg; see state/types.ts's own RULE at the top of the file for why this is called fresh on every render rather than cached), fed straight into that format's own readXContent. markdown has a live-view MarkdownEditor but no package at all -- `readMarkdownContent` runs on `doc.editor.toMarkdownText()` (re-serialised fresh, reflecting in-progress edits, matching export-pdf.ts's own convention), the same role `.editor.toPackage()` plays elsewhere. pdf, xlsx, csv, svg, rtf, and wpd all already carry a LayoutMetadata directly on `.layout` (each read-only preview format's own `.layout` is the throwaway to-Pdf-then-readPdf conversion open-document.ts already computed at open time -- see that module's own XlsxOpenDocument/CsvOpenDocument/SvgOpenDocument/RtfOpenDocument/WpdOpenDocument doc comments), so none of them needs a read call here at all. odb has no document-level metadata concept anywhere in this codebase (it is a table/form/report container, not a single document with its own title/author/etc.) -- this throws rather than fabricating an empty LayoutMetadata, and the metadata screen itself is the one place that catches it and shows a plain message instead of crashing. export function metadataFor(doc: OpenDocument): LayoutMetadata { switch (doc.format) { case "docx": @@ -32,6 +32,7 @@ export function metadataFor(doc: OpenDocument): LayoutMetadata { case "csv": case "svg": case "rtf": + case "wpd": return doc.layout.metadata; case "odb": throw new Error( diff --git a/packages/document-cli/src/tui/screens/editors/pdf/shared.ts b/packages/document-cli/src/tui/screens/editors/pdf/shared.ts index c1c277d10..8573e21ed 100644 --- a/packages/document-cli/src/tui/screens/editors/pdf/shared.ts +++ b/packages/document-cli/src/tui/screens/editors/pdf/shared.ts @@ -5,6 +5,7 @@ import type { PdfOpenDocument, RtfOpenDocument, SvgOpenDocument, + WpdOpenDocument, XlsxOpenDocument, } from "../../../state/types.js"; import { @@ -15,7 +16,7 @@ import { export { parseColorField, parseStrokeField }; export { parseNumberField } from "../../shared/text.js"; -// Every screen in this directory is only ever reached from `pdfPageList`, the root screen `rootScreenForFormat` produces for an open PDF document or for one of the four formats opened read-only as a converted PDF preview (an xlsx workbook, a csv sheet, an svg drawing, an rtf document -- see state/types.ts's own XlsxOpenDocument/CsvOpenDocument/SvgOpenDocument/RtfOpenDocument doc comments) -- so `state.openDocument` is always one of these five by the time any screen here renders. All five carry the identical `.layout: LayoutDocument` field this whole screen group reads from, and nothing else, which is exactly what lets one screen family serve all of them with no per-format branch anywhere in page-list.tsx/page-items.tsx/item-detail.tsx. This throws rather than falling back to an empty view because a mismatch would mean the app router itself is broken, not a recoverable, user-facing condition. +// Every screen in this directory is only ever reached from `pdfPageList`, the root screen `rootScreenForFormat` produces for an open PDF document or for one of the five formats opened read-only as a converted PDF preview (an xlsx workbook, a csv sheet, an svg drawing, an rtf document, a wpd document -- see state/types.ts's own XlsxOpenDocument/CsvOpenDocument/SvgOpenDocument/RtfOpenDocument/WpdOpenDocument doc comments) -- so `state.openDocument` is always one of these six by the time any screen here renders. All six carry the identical `.layout: LayoutDocument` field this whole screen group reads from, and nothing else, which is exactly what lets one screen family serve all of them with no per-format branch anywhere in page-list.tsx/page-items.tsx/item-detail.tsx. This throws rather than falling back to an empty view because a mismatch would mean the app router itself is broken, not a recoverable, user-facing condition. export function requirePdfDocument( openDocument: OpenDocument | undefined, ): @@ -23,29 +24,32 @@ export function requirePdfDocument( | XlsxOpenDocument | CsvOpenDocument | SvgOpenDocument - | RtfOpenDocument { + | RtfOpenDocument + | WpdOpenDocument { if ( openDocument?.format !== "pdf" && openDocument?.format !== "xlsx" && openDocument?.format !== "csv" && openDocument?.format !== "svg" && - openDocument?.format !== "rtf" + openDocument?.format !== "rtf" && + openDocument?.format !== "wpd" ) { throw new Error( - "A PDF inspection screen rendered without an open PDF, xlsx, csv, svg, or rtf document; the app router only reaches this screen group from pdfPageList, which is only ever the root screen of one of those five formats.", + "A PDF inspection screen rendered without an open PDF, xlsx, csv, svg, rtf, or wpd document; the app router only reaches this screen group from pdfPageList, which is only ever the root screen of one of those six formats.", ); } return openDocument; } -// The editing-capable narrowing of the above: an xlsx workbook, csv sheet, svg drawing, or rtf document opens as a fixed, one-shot PDF preview with no live `PdfEditor` behind it at all (see those formats' own OpenDocument doc comments -- each carries `layout`/`bytes`, never an `editor`), so add/edit/delete only ever make sense for a genuine `'pdf'`-format document. Screens call this only from the code paths that mutate (the add-item flow, item-detail's field editor); the plain read-only list/dump views keep using `requirePdfDocument` above so an opened preview format still browses exactly like a real PDF. +// The editing-capable narrowing of the above: an xlsx workbook, csv sheet, svg drawing, rtf document, or wpd document opens as a fixed, one-shot PDF preview with no live `PdfEditor` behind it at all (see those formats' own OpenDocument doc comments -- each carries `layout`/`bytes`, never an `editor`), so add/edit/delete only ever make sense for a genuine `'pdf'`-format document. Screens call this only from the code paths that mutate (the add-item flow, item-detail's field editor); the plain read-only list/dump views keep using `requirePdfDocument` above so an opened preview format still browses exactly like a real PDF. export function isEditablePdfDocument( doc: | PdfOpenDocument | XlsxOpenDocument | CsvOpenDocument | SvgOpenDocument - | RtfOpenDocument, + | RtfOpenDocument + | WpdOpenDocument, ): doc is PdfOpenDocument { return doc.format === "pdf"; } diff --git a/packages/document-cli/src/tui/state/reducer.ts b/packages/document-cli/src/tui/state/reducer.ts index 078874ebe..b373cf80f 100644 --- a/packages/document-cli/src/tui/state/reducer.ts +++ b/packages/document-cli/src/tui/state/reducer.ts @@ -171,6 +171,8 @@ function documentWithPath(doc: OpenDocument, path: string): OpenDocument { return { format: "svg", layout: doc.layout, bytes: doc.bytes, path }; case "rtf": return { format: "rtf", layout: doc.layout, bytes: doc.bytes, path }; + case "wpd": + return { format: "wpd", layout: doc.layout, bytes: doc.bytes, path }; } } @@ -806,11 +808,12 @@ export function appReducer(state: AppState, action: Action): AppState { stack: [rootScreenForFormat(action.doc.format)], }, "info", - // xlsx, csv, svg, and rtf have no editor to open at all -- action.doc is already a read-only PDF-preview conversion by the time it reaches here (see format/open-document.ts) -- so these are the formats whose "opened" message doubles as pointing the way to the one thing that can actually be done with them next. + // xlsx, csv, svg, rtf, and wpd have no editor to open at all -- action.doc is already a read-only PDF-preview conversion by the time it reaches here (see format/open-document.ts) -- so these are the formats whose "opened" message doubles as pointing the way to the one thing that can actually be done with them next. action.doc.format === "xlsx" || action.doc.format === "csv" || action.doc.format === "svg" || - action.doc.format === "rtf" + action.doc.format === "rtf" || + action.doc.format === "wpd" ? `Opened ${action.path} as a read-only PDF preview -- press ':' then 'export pdf' to save it as a real PDF` : `Opened ${action.path}`, ); @@ -2041,7 +2044,8 @@ export function appReducer(state: AppState, action: Action): AppState { doc.format === "xlsx" || doc.format === "csv" || doc.format === "svg" || - doc.format === "rtf" + doc.format === "rtf" || + doc.format === "wpd" ) { return withStatus( state, diff --git a/packages/document-cli/src/tui/state/types.ts b/packages/document-cli/src/tui/state/types.ts index b896abab5..d44579ae3 100644 --- a/packages/document-cli/src/tui/state/types.ts +++ b/packages/document-cli/src/tui/state/types.ts @@ -184,6 +184,14 @@ export interface RtfOpenDocument { readonly path: string; } +// wpd mirrors rtf: no editor -- wpd-codec is read-only, so there is no XmlElement tree and could never be a live view even in principle -- but a genuine `wpd -> pdf` layout-engine edge (documents.js's own composition engine, hasLayoutPath: true), opened read-only as its own `readPdf` result through the shared pdf screen family. Unlike rtf, there is no `wpdToPdf` named convenience function (wpd has no reverse direction to pair it with, and documents.js's own README states named forwarders are ergonomic sugar over `convertDocument`, not a requirement of a pair being routable), so `open-document.ts` reaches the identical edge through `convertDocument("wpd", "pdf", bytes)` directly. +export interface WpdOpenDocument { + readonly format: "wpd"; + readonly layout: LayoutDocument; + readonly bytes: Uint8Array; + readonly path: string; +} + // The seven formats that have a live-view editor, and therefore support every mutating action, `editor.toBytes()` saving, undo snapshots. `odb`/`xlsx`/`csv`/`svg`/`rtf` are read-only sources; `pdf` joined this union once documents.js gained a real live-view `PdfEditor` -- see PdfOpenDocument's own doc comment. `pdf` is deliberately excluded from exportToPdf's own conversion set even though it is editable now: there is no docxToPdf-equivalent "convert a PDF to a PDF" function, and there does not need to be one -- editing and saving a PDF in place needs no conversion step at all. export type EditableOpenDocument = | DocxOpenDocument @@ -203,7 +211,8 @@ export type OpenDocument = | XlsxOpenDocument | CsvOpenDocument | SvgOpenDocument - | RtfOpenDocument; + | RtfOpenDocument + | WpdOpenDocument; export type EditableFormat = EditableOpenDocument["format"]; @@ -390,6 +399,7 @@ export function rootScreenForFormat(format: OpenDocumentFormat): Screen { case "csv": case "svg": case "rtf": + case "wpd": return { kind: "pdfPageList" }; } } diff --git a/packages/document-mcp/package.json b/packages/document-mcp/package.json index a2ef5d035..e9c51d33c 100644 --- a/packages/document-mcp/package.json +++ b/packages/document-mcp/package.json @@ -105,6 +105,7 @@ "tsdown": "^0.22.14", "turbo": "^2.10.8", "typescript": "^6.0.3", - "vitest": "^4.1.10" + "vitest": "^4.1.10", + "wpd-codec": "^1.0.1" } } diff --git a/packages/document-mcp/src/io/document-input.ts b/packages/document-mcp/src/io/document-input.ts index dfa99c1fa..eba0266bc 100644 --- a/packages/document-mcp/src/io/document-input.ts +++ b/packages/document-mcp/src/io/document-input.ts @@ -30,6 +30,7 @@ const EXTENSION_TO_FORMAT: Readonly> = { markdown: "markdown", md: "markdown", rtf: "rtf", + wpd: "wpd", csv: "csv", svg: "svg", pdf: "pdf", diff --git a/packages/document-mcp/src/test-support/wpd-fixture.ts b/packages/document-mcp/src/test-support/wpd-fixture.ts new file mode 100644 index 000000000..7891848c1 --- /dev/null +++ b/packages/document-mcp/src/test-support/wpd-fixture.ts @@ -0,0 +1,53 @@ +// A minimal but structurally authentic WordPerfect 6.x file: a real 512-byte prefix header, a one-record (index-header-only) index area, no packets, and a document area built from real WPFF Document Structure function bytes -- mirroring this package's own src/test-support/odf-formula-fixture.ts convention (that file hand-authors odf's own byte structure since neither odf.js nor documents.js exposes a writer for it; wpd-codec exposes no writer at all, for the identical reason). documents.js's own internal src/test-support/wpd.ts carries the same construction (never exported, so not reusable directly from this package either) -- this is that same small, spec-grounded port, not a new design. +import { WPD_INDEX_RECORD_SIZE } from "wpd-codec/container/prefix"; + +const PREFIX_HEADER_SIZE = 512; +const FILE_ID = [0xff, 0x57, 0x50, 0x43]; + +function putUint16(bytes: Uint8Array, offset: number, value: number): void { + bytes[offset] = value & 0xff; + bytes[offset + 1] = (value >>> 8) & 0xff; +} + +function putUint32(bytes: Uint8Array, offset: number, value: number): void { + putUint16(bytes, offset, value & 0xffff); + putUint16(bytes, offset + 2, (value >>> 16) & 0xffff); +} + +// The ASCII characters of a string as document-area bytes -- every character in the single-byte printable range passes through unchanged except a space, which WordPerfect represents as the Soft Space function (0x80) rather than byte 0x20 (the international shorthand for the sharp s). +function documentAreaText(value: string): number[] { + return [...value].map((character) => { + const code = character.charCodeAt(0); + return code === 0x20 ? 0x80 : code; + }); +} + +function buildPacketFreeWpdFile(documentArea: readonly number[]): Uint8Array { + const documentAreaStart = PREFIX_HEADER_SIZE + WPD_INDEX_RECORD_SIZE; // one record: the index header, no packets. + const fileSize = documentAreaStart + documentArea.length; + + const bytes = new Uint8Array(fileSize); + bytes.set(FILE_ID, 0); + putUint32(bytes, 4, documentAreaStart); + bytes[8] = 1; // product type: WordPerfect + bytes[9] = 0x0a; // file type: WordPerfect document + bytes[10] = 2; // major version: the 6.x-X6 lineage + bytes[11] = 1; // minor version + putUint16(bytes, 12, 0); // not encrypted + putUint16(bytes, 14, PREFIX_HEADER_SIZE); // pointer to the index area + putUint32(bytes, 16, 5); // the documented reserved long at the head of the extended header + putUint32(bytes, 20, fileSize); + + bytes[PREFIX_HEADER_SIZE] = 2; // index header flags + putUint16(bytes, PREFIX_HEADER_SIZE + 2, 1); // record count: the index header alone + + bytes.set(documentArea, documentAreaStart); + return bytes; +} + +// One real paragraph of ordinary body text, matching this package's own docx/odt fixtures in buildFormatFixtures(). +export function wpdFixtureBytes(): Uint8Array { + return buildPacketFreeWpdFile( + documentAreaText("A paragraph of ordinary body text."), + ) as Uint8Array; +} diff --git a/packages/document-mcp/src/tools/outline.test.ts b/packages/document-mcp/src/tools/outline.test.ts index 1d829edbe..6a3e9b04d 100644 --- a/packages/document-mcp/src/tools/outline.test.ts +++ b/packages/document-mcp/src/tools/outline.test.ts @@ -18,6 +18,7 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { createServer } from "../server"; import { odfFormulaBytes } from "../test-support/odf-formula-fixture"; import { buildMultiPagePdf } from "../test-support/pdf-fixture"; +import { wpdFixtureBytes } from "../test-support/wpd-fixture"; // Drives the real, fully-assembled MCP server (createServer(), the same entry point src/bin.ts uses) through a genuine in-memory client/server JSON-RPC round trip -- proving `outline_document` is registered under that name, reads a real source document's own native tree directly (documents.js's readNativeDocumentTree, no bridging conversion involved), and answers with document-outline.js's buildOutline TOC projection as structured JSON an MCP client can render directly. @@ -247,6 +248,7 @@ function buildFormatFixtures(): Record< kind: "wordprocessing", }, svg: { bytes: odgToSvg(singlePageOdgBytes), kind: "drawing" }, + wpd: { bytes: wpdFixtureBytes(), kind: "wordprocessing" }, xlsx: { bytes: xlsxBytes, kind: "spreadsheet" }, }; } diff --git a/packages/documents.js/README.md b/packages/documents.js/README.md index 8d39b84b1..2f04988c0 100644 --- a/packages/documents.js/README.md +++ b/packages/documents.js/README.md @@ -2,7 +2,7 @@ [![GitHub](https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white)](https://github.com/ExaDev/documents.js/tree/main/packages/documents.js) [![npm](https://img.shields.io/badge/npm-CB3837?logo=npm&logoColor=white)](https://www.npmjs.com/package/documents.js) [![npm version](https://img.shields.io/npm/v/documents.js)](https://www.npmjs.com/package/documents.js) [![CI](https://img.shields.io/github/actions/workflow/status/ExaDev/documents.js/ci.yml?branch=main)](https://github.com/ExaDev/documents.js/actions) -> Converts between any two compatible document formats through a shared content/layout pivot. docx, pptx, odt, odp, ods, odg, xlsx, csv (TSV is the same format with a tab delimiter), svg, markdown, and rtf all read into and build from the same shared `ContentDocument` model (reported to callers as the tree-form `DocumentTree`), with PDF — reached through pdf-codec's own `LayoutDocument` view — as the one format every variant can reach. A composition engine (`convertDocument`) routes 129 (source, target) pairs across the eleven content formats and PDF, including twenty-two PDF-pivot round trips (the eight layout-engine formats, plus xlsx and csv composing through ods, and rtf composing through docx/odt/markdown), twenty-four cross-format bridge functions (same-variant direct copies, cross-variant semantic transforms, and PDF-composed), plus special-case conversions for `.odm` master documents, `.odb` database front-ends (HSQLDB and Firebird, four storage tiers), standalone `.odf` formula documents, and a bounded SQL/rpt-formula engine for `.odb` reports. Also includes: read-and-write live-view editors for all six editable formats, docx comment/footnote/header-footer exposure via `readDocxExtras`, real font resolution (source-embedded faces ahead of caller-supplied, vendored substitutes, and the standard 14), a hand-written MathML typesetting engine with embedded-font PDF rendering and a matching MathML ⇄ OMML translator, LaTeX lowering into the schema's two-layer semantic math core (pinned temml parser, symbol tables from prose, a coherence lint), and a fully hand-written PDF codec. Built on [ooxml.js](../ooxml.js/README.md), [odf.js](../odf.js/README.md), [pdf-codec](../pdf-codec/README.md), [markdown-codec](../markdown-codec/README.md), [rtf-codec](../rtf-codec/README.md), and [document-schema.js](../document-schema.js/README.md). +> Converts between any two compatible document formats through a shared content/layout pivot. docx, pptx, odt, odp, ods, odg, xlsx, csv (TSV is the same format with a tab delimiter), svg, markdown, and rtf all read into and build from the same shared `ContentDocument` model (reported to callers as the tree-form `DocumentTree`), with PDF — reached through pdf-codec's own `LayoutDocument` view — as the one format every variant can reach; wpd (WordPerfect 6.x-X6) reads into the same wordprocessing variant as a read-only source, routable everywhere the others are but never buildable as a target, since wpd-codec ships no writer. A composition engine (`convertDocument`) routes 141 (source, target) pairs across the twelve content formats and PDF, including twenty-two PDF-pivot round trips (the eight layout-engine formats, plus xlsx and csv composing through ods, and rtf composing through docx/odt/markdown), twenty-four cross-format bridge functions (same-variant direct copies, cross-variant semantic transforms, and PDF-composed), twelve one-way wpd-sourced routes, plus special-case conversions for `.odm` master documents, `.odb` database front-ends (HSQLDB and Firebird, four storage tiers), standalone `.odf` formula documents, and a bounded SQL/rpt-formula engine for `.odb` reports. Also includes: read-and-write live-view editors for all six editable formats, docx comment/footnote/header-footer exposure via `readDocxExtras`, real font resolution (source-embedded faces ahead of caller-supplied, vendored substitutes, and the standard 14), a hand-written MathML typesetting engine with embedded-font PDF rendering and a matching MathML ⇄ OMML translator, LaTeX lowering into the schema's two-layer semantic math core (pinned temml parser, symbol tables from prose, a coherence lint), and a fully hand-written PDF codec. Built on [ooxml.js](../ooxml.js/README.md), [odf.js](../odf.js/README.md), [pdf-codec](../pdf-codec/README.md), [markdown-codec](../markdown-codec/README.md), [rtf-codec](../rtf-codec/README.md), [wpd-codec](../wpd-codec/README.md), and [document-schema.js](../document-schema.js/README.md). `documents.js` extends `ooxml.js` in two directions `ooxml.js` deliberately does not cover: full PDF support (parsing and generating, via `pdf-codec`), and a read-**and-write** manipulation API for docx/pptx content — `ooxml.js`'s own typed readers are one-way. The PDF codec is hand-written against ISO 32000-1, with no external PDF library as a dependency — see [Fidelity](#fidelity) and pdf-codec's own README for the honest trade-off (not as robust against adversarial PDFs as a 15+-year-hardened library; fully auditable and dependency-free instead). `src/mathml/` (the MathML typesetting engine) stays in this package and is hand-written too, for the same supply-chain reason. The one deliberate exception on the math side is the LaTeX parser: `src/latex/` lowers LaTeX into the schema's semantic core over a pinned exact-version [temml](https://temml.org) dependency — see [LaTeX lowering into the semantic core](#latex-lowering-into-the-semantic-core) for why a LaTeX grammar is the one component not worth hand-writing and what the pin guarantees. @@ -14,6 +14,7 @@ graph TD pdfcodec("pdf-codec") mdcodec("markdown-codec") rtfcodec("rtf-codec") + wpdcodec("wpd-codec") bytecodec("byte-codec") documents("documents.js") mcp("document-mcp") @@ -24,12 +25,14 @@ graph TD schema --> pdfcodec schema --> mdcodec schema --> rtfcodec + schema --> wpdcodec schema --> documents ooxml --> documents odf --> documents pdfcodec --> documents mdcodec --> documents rtfcodec --> documents + wpdcodec --> documents bytecodec --> pdfcodec bytecodec --> documents documents --> mcp @@ -44,6 +47,7 @@ graph TD click pdfcodec "https://github.com/ExaDev/documents.js/tree/main/packages/pdf-codec" "pdf-codec" click mdcodec "https://github.com/ExaDev/documents.js/tree/main/packages/markdown-codec" "markdown-codec" click rtfcodec "https://github.com/ExaDev/documents.js/tree/main/packages/rtf-codec" "rtf-codec" + click wpdcodec "https://github.com/ExaDev/documents.js/tree/main/packages/wpd-codec" "wpd-codec" click bytecodec "https://github.com/ExaDev/documents.js/tree/main/packages/byte-codec" "byte-codec" click documents "https://github.com/ExaDev/documents.js" "documents.js" click mcp "https://github.com/ExaDev/documents.js/tree/main/packages/document-mcp" "document-mcp" @@ -76,7 +80,7 @@ npm install documents.js document-schema.js ### The generic entry point: `convertDocument` -A single function, `convertDocument`, sits behind every named conversion and reaches every pair the composition engine can route — all 129 supported (source, target) combinations. The named functions below are thin one-line forwarders to it; they remain the ergonomic layer for a caller who wants a fixed pair and autocomplete discovery, while `convertDocument` is the first-class entry point for a caller working from a runtime format pair (CLI, MCP tool, matrix enumeration). +A single function, `convertDocument`, sits behind every named conversion and reaches every pair the composition engine can route — all 141 supported (source, target) combinations. The named functions below are thin one-line forwarders to it; they remain the ergonomic layer for a caller who wants a fixed pair and autocomplete discovery, while `convertDocument` is the first-class entry point for a caller working from a runtime format pair (CLI, MCP tool, matrix enumeration). ```ts import { convertDocument } from "documents.js"; @@ -196,14 +200,14 @@ const { document, diagnostics } = await converter.convert( ); ``` -`DocumentFormat` includes `docx`/`pptx`/`xlsx`/`odt`/`odp`/`ods`/`odg`/`svg`/`odf`/`csv`/`markdown`/`rtf`/`pdf` — thirteen members. The port's `conversions` list is derived from `resolveCompositionPlan` plus the `odf`→`pdf` special case — 129 pairs total. `DocumentFormat` is inferred from `DocumentFormatSchema` (a real Zod schema); `DOCUMENT_FORMATS` is exported as a plain array derived from the same schema: +`DocumentFormat` includes `docx`/`pptx`/`xlsx`/`odt`/`odp`/`ods`/`odg`/`svg`/`odf`/`csv`/`markdown`/`rtf`/`wpd`/`pdf` — fourteen members, `wpd` the one read-only member: it appears as a source in `conversions` but never as a target, since wpd-codec ships no writer. The port's `conversions` list is derived from `resolveCompositionPlan` plus the `odf`→`pdf` special case — 141 pairs total. `DocumentFormat` is inferred from `DocumentFormatSchema` (a real Zod schema); `DOCUMENT_FORMATS` is exported as a plain array derived from the same schema: The port also exposes `contractVersion: number`, bumped only when `DocumentConverter`'s own contract shape changes — a new field on `ConversionResult` a caller might need to branch on, or a new `ConversionOptions` field an implementation is now expected to honour — never when the `conversions` table simply grows with more supported source/target pairs (that's discoverable at runtime via `conversions` itself). It is currently `7`: the bump from `6` reflects `ConversionResult.package` changing type to the tree-form `DocumentTree` described below, which a caller reading that field must now flatten rather than read directly. ```ts import { DOCUMENT_FORMATS, DocumentFormatSchema } from "documents.js"; -console.log(DOCUMENT_FORMATS); // ['docx', 'pptx', 'xlsx', 'odt', 'odp', 'ods', 'odg', 'svg', 'odf', 'csv', 'markdown', 'rtf', 'pdf'] +console.log(DOCUMENT_FORMATS); // ['docx', 'pptx', 'xlsx', 'odt', 'odp', 'ods', 'odg', 'svg', 'odf', 'csv', 'markdown', 'rtf', 'wpd', 'pdf'] DocumentFormatSchema.parse(userSuppliedFormat); // throws a ZodError for anything outside that list ``` @@ -770,7 +774,7 @@ The package is layered from generic primitives outward to the two conversion dir - **`src/metadata/`** — cross-format metadata read/write via `DOCUMENT_FORMAT_CODECS`. - **`src/package-codec.ts`** — `decodeDocumentPackage`/`encodeDocumentPackage`/`decodeOdbPackage`. -Dependency direction is downward and checkable. Seven external dependencies each own a distinct concern: `ooxml.js` (docx/pptx/xlsx), `odf.js` (odt/ods/odp/odg), `document-schema.js` (shared schemas + port contracts), `pdf-codec` (PDF codec + text-layout/font primitives), `byte-codec` (byte/image utilities), `markdown-codec` (markdown), `rtf-codec` (rtf). No `PdfObject`/`PdfDict`/`PdfStream` type appears anywhere in this package. +Dependency direction is downward and checkable. Eight external dependencies each own a distinct concern: `ooxml.js` (docx/pptx/xlsx), `odf.js` (odt/ods/odp/odg), `document-schema.js` (shared schemas + port contracts), `pdf-codec` (PDF codec + text-layout/font primitives), `byte-codec` (byte/image utilities), `markdown-codec` (markdown), `rtf-codec` (rtf), `wpd-codec` (wpd, read-only). No `PdfObject`/`PdfDict`/`PdfStream` type appears anywhere in this package. ## Build, test, and lint @@ -883,25 +887,26 @@ To run a single test file: `pnpm vitest run src/path/to/file.test.ts`. ## Fidelity -Read as **row → column**. `✓` lossless, `~` bounded, `✗` lossy, `✗✗` severe, `→` one-way, `–` no conversion. `.odm`/`.odb` sit outside this table. `rtf` is wired into the composition engine and bidirectionally routable to every other format here except `csv`/`xlsx` (one hop past the pathfinder's own 3-hop cap) and `odf` (excluded from routing entirely — see below). Every cell below is derived from `resolveCompositionPlan`'s actual resolved route for that pair (hop count and hop kind — same-variant bridge, cross-variant transform, or PDF pivot), not hand-reasoned: the pathfinder generates the full cross-product of same-variant and cross-variant-transform edges, so a pair can be routable even with no named convenience function for it (e.g. `odp → docx`, `odt → pptx`). - -| ↓ from \ to → | docx | pptx | xlsx | odt | odp | ods | odg | svg | odf | markdown | csv | rtf | pdf | -| ------------- | ---- | ---- | ---- | --- | --- | --- | --- | --- | --- | -------- | --- | --- | --- | -| **docx** | — | ~ | ✗ | ✓ | ~ | ✗ | ✗ | ✗ | – | ✗ | ✗ | ✗ | ~ | -| **pptx** | ~ | — | ✗ | ~ | ✓ | ✗ | ~ | ~ | – | ~ | ✗ | ~ | ~ | -| **xlsx** | ✗ | ✗ | — | ✗ | ✗ | ~ | ✗ | ✗ | – | ✗✗ | ~ | – | ~ | -| **odt** | ✓ | ~ | ✗ | — | ~ | ✗ | ✗ | ✗ | – | ✗ | ✗ | ✗ | ~ | -| **odp** | ~ | ✓ | ✗ | ~ | — | ✗ | ~ | ~ | – | ~ | ✗ | ~ | ~ | -| **ods** | ✗ | ✗ | ~ | ✗ | ✗ | — | ✗ | ✗ | – | ✗✗ | ~ | ✗ | ~ | -| **odg** | ✗ | ~ | ✗ | ✗ | ~ | ✗ | — | ✓ | – | ✗ | ✗ | ✗ | ~ | -| **svg** | ✗ | ~ | ✗ | ✗ | ~ | ✗ | ✓ | — | – | ✗✗ | ✗✗ | ✗ | ~ | -| **odf** | – | – | – | – | – | – | – | – | — | – | – | – | → | -| **markdown** | ~ | ~ | ✗✗ | ~ | ~ | ✗ | ✗ | ✗✗ | – | — | ✗✗ | ~ | ~ | -| **csv** | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | – | ✗✗ | — | – | ~ | -| **rtf** | ~ | ~ | – | ~ | ~ | ✗ | ✗ | ✗ | – | ✗ | – | — | ~ | -| **pdf** | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – | ✗✗ | ✗ | ✗ | — | - -129 of 156 directional pairs are routable. The shared `ContentDocument` model is the hub, not PDF — twenty bridges bypass PDF entirely. +Read as **row → column**. `✓` lossless, `~` bounded, `✗` lossy, `✗✗` severe, `→` one-way, `–` no conversion. `.odm`/`.odb` sit outside this table. `rtf` is wired into the composition engine and bidirectionally routable to every other format here except `csv`/`xlsx` (one hop past the pathfinder's own 3-hop cap) and `odf` (excluded from routing entirely — see below). `wpd` is wired in as a read-only source (see composition.ts's own `ReadOnlyContentFormat`): routable to every other format here except `odf`, exactly like `rtf`'s reach, but every one of its cells is `→` rather than a fidelity grade, since a read-only format has no reverse direction to compare against and no column of its own — nothing ever routes to `wpd`, wpd-codec having no writer at all. Every cell below is derived from `resolveCompositionPlan`'s actual resolved route for that pair (hop count and hop kind — same-variant bridge, cross-variant transform, or PDF pivot), not hand-reasoned: the pathfinder generates the full cross-product of same-variant and cross-variant-transform edges, so a pair can be routable even with no named convenience function for it (e.g. `odp → docx`, `odt → pptx`). + +| ↓ from \ to → | docx | pptx | xlsx | odt | odp | ods | odg | svg | odf | markdown | csv | rtf | wpd | pdf | +| ------------- | ---- | ---- | ---- | --- | --- | --- | --- | --- | --- | -------- | --- | --- | --- | --- | +| **docx** | — | ~ | ✗ | ✓ | ~ | ✗ | ✗ | ✗ | – | ✗ | ✗ | ✗ | – | ~ | +| **pptx** | ~ | — | ✗ | ~ | ✓ | ✗ | ~ | ~ | – | ~ | ✗ | ~ | – | ~ | +| **xlsx** | ✗ | ✗ | — | ✗ | ✗ | ~ | ✗ | ✗ | – | ✗✗ | ~ | – | – | ~ | +| **odt** | ✓ | ~ | ✗ | — | ~ | ✗ | ✗ | ✗ | – | ✗ | ✗ | ✗ | – | ~ | +| **odp** | ~ | ✓ | ✗ | ~ | — | ✗ | ~ | ~ | – | ~ | ✗ | ~ | – | ~ | +| **ods** | ✗ | ✗ | ~ | ✗ | ✗ | — | ✗ | ✗ | – | ✗✗ | ~ | ✗ | – | ~ | +| **odg** | ✗ | ~ | ✗ | ✗ | ~ | ✗ | — | ✓ | – | ✗ | ✗ | ✗ | – | ~ | +| **svg** | ✗ | ~ | ✗ | ✗ | ~ | ✗ | ✓ | — | – | ✗✗ | ✗✗ | ✗ | – | ~ | +| **odf** | – | – | – | – | – | – | – | – | — | – | – | – | – | → | +| **markdown** | ~ | ~ | ✗✗ | ~ | ~ | ✗ | ✗ | ✗✗ | – | — | ✗✗ | ~ | – | ~ | +| **csv** | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | – | ✗✗ | — | – | – | ~ | +| **rtf** | ~ | ~ | – | ~ | ~ | ✗ | ✗ | ✗ | – | ✗ | – | — | – | ~ | +| **wpd** | → | → | → | → | → | → | → | → | – | → | → | → | — | → | +| **pdf** | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | – | ✗✗ | ✗ | ✗ | – | — | + +141 of 182 directional pairs are routable. The shared `ContentDocument` model is the hub, not PDF — twenty bridges bypass PDF entirely. **X → PDF** is a genuine layout render: positioned text, images, tables, lists, vector primitives, styled through the full cascade. It is a faithful visual approximation, not pixel-identical — closeness depends on font availability. @@ -923,6 +928,8 @@ Read as **row → column**. `✓` lossless, `~` bounded, `✗` lossy, `✗✗` s **Every wordprocessing-family format (docx/odt/markdown/rtf) bridges to every presentation-family format (pptx/odp), and every drawing-family format (odg/svg) bridges to every presentation-family format too, through a semantic transform** — not just the two hand-written convenience pairs (docx⇄pptx, odt⇄odp): the pathfinder generates the full cross-product from the same two transform functions (`wordprocessingToPresentation`/`presentationToWordprocessing`, `drawingToPresentation`/`presentationToDrawing`), so e.g. `odp → docx` and `odt → pptx` are real routable pairs with no named function of their own. Slide boundaries are heuristic, but blocks survive intact. A wordprocessing format reaching a drawing format (or vice versa) chains two such transforms through presentation as a hub (e.g. `docx → pptx → odg`) with no PDF pivot at all — cheaper than a PDF round trip, but compounding two structural transforms is more lossy than one, hence those cells' `✗` rather than `~`. +**`wpd`'s whole row is `→`, never a fidelity grade, because it is read-only.** wpd-codec has a real reader and no writer at all (a deliberate scope decision, not an omission — see that package's own Scope), so there is no reverse conversion to measure loss against the way every other pair in this table can be. `wpd → docx`/`wpd → odt`/`wpd → markdown`/`wpd → rtf` are same-variant bridges at cost 1; `wpd → pptx`/`wpd → odp` are cross-variant transforms at cost 2; `wpd → pdf` is a direct layout-engine pass, the same edge markdown's own `hasLayoutPath` justifies; everything else composes through one of those. `wpd → odf` is the one cell excluded, matching every other row: nothing routes to `odf`. + **`.odb` extraction** is genuine verified data extraction across all four tiers, differing by what each storage shape carries. BLOB content recovers byte-for-byte. No reverse direction. **SQL/rpt engines** are exact within their closed grammars, hard failures outside — never approximations. diff --git a/packages/documents.js/package.json b/packages/documents.js/package.json index 38323f5b5..37a4803cf 100644 --- a/packages/documents.js/package.json +++ b/packages/documents.js/package.json @@ -109,6 +109,7 @@ "pdf-codec": "^3.6.2", "rtf-codec": "^1.1.0", "temml": "0.13.4", + "wpd-codec": "workspace:^", "zod": "^4.4.3" }, "devDependencies": { diff --git a/packages/documents.js/src/codecs/read.ts b/packages/documents.js/src/codecs/read.ts index b2ab7315a..0a2752725 100644 --- a/packages/documents.js/src/codecs/read.ts +++ b/packages/documents.js/src/codecs/read.ts @@ -11,6 +11,7 @@ import { readCsvContent } from "../csv/read"; import { decodeSvgText } from "../svg/text"; import { readSvgContent } from "../svg/read"; import { readRtfContent } from "rtf-codec"; +import { readWpdContent } from "wpd-codec"; import { readOdfFormulaContent } from "../odf/formula/read"; import { readOdgContent } from "../odf/odg/read"; import { readOdpContent } from "../odf/odp/read"; @@ -102,6 +103,11 @@ export const CONTENT_READERS: Readonly< decodeDocumentPackage("xlsx", requireArrayBufferBytes(bytes)), ); }, + // readWpdContent takes bytes directly -- a WordPerfect file is a prefix and a function-code stream, not a package this workspace's own decodeDocumentPackage knows, and its own container detection (a bare file versus an OLE compound wrapper) happens inside the reader. It has no loop of its own to hook a signal into, so its read checks the signal once before decoding, the shape every no-package format above gets from throwIfAborted. There is no matching entry in the registry's write half at all: wpd-codec ships no writer, which is what makes wpd a read-only format everywhere else in this package. + wpd: (bytes, options) => { + throwIfAborted(options?.signal); + return readWpdContent(bytes); + }, }; // pdf's own read half, also moved verbatim from the registry: readPdf from 'pdf-codec/read' (never the root barrel -- see the module comment), forwarding the signal since readPdf has a page loop of its own. diff --git a/packages/documents.js/src/codecs/registry.ts b/packages/documents.js/src/codecs/registry.ts index b9c9308be..d17a5e1b7 100644 --- a/packages/documents.js/src/codecs/registry.ts +++ b/packages/documents.js/src/codecs/registry.ts @@ -128,4 +128,10 @@ export const DOCUMENT_FORMAT_CODECS: Readonly< encodeDocumentPackage("xlsx", buildXlsxPackageFromContent(content)), }, }, + // wpd has a content.read and no content.write, exactly as odf does, and for a directly analogous reason: wpd-codec ships no writer at all. The two differ only in why -- odf has no ContentDocument-to-formula-document path anywhere in the family, while wpd's absent writer is a scope decision wpd-codec states itself -- and both surface here as an entry whose `write` is left unset rather than stubbed, so every consumer that needs to build bytes discovers the absence through the same missing field. + wpd: { + content: { + read: CONTENT_READERS.wpd, + }, + }, }; diff --git a/packages/documents.js/src/convert/capability.test.ts b/packages/documents.js/src/convert/capability.test.ts index 02b352c0e..362c3d0c2 100644 --- a/packages/documents.js/src/convert/capability.test.ts +++ b/packages/documents.js/src/convert/capability.test.ts @@ -16,7 +16,7 @@ describe("FORMAT_CAPABILITIES", () => { } expect(new Set(byVariant.get("wordprocessing"))).toEqual( - new Set(["docx", "odt", "markdown", "rtf"]), + new Set(["docx", "odt", "markdown", "rtf", "wpd"]), ); expect(new Set(byVariant.get("presentation"))).toEqual( new Set(["pptx", "odp"]), diff --git a/packages/documents.js/src/convert/capability.ts b/packages/documents.js/src/convert/capability.ts index f81505de5..1e09af7b4 100644 --- a/packages/documents.js/src/convert/capability.ts +++ b/packages/documents.js/src/convert/capability.ts @@ -12,36 +12,108 @@ export interface FormatCapability { readonly variant?: ContentVariant; // Whether a direct layout-engine conversion (a real ContentDocument -> LayoutDocument -> PDF pipeline, or its reverse) already exists for this format today. `odf` is a one-way exception -- formula -> PDF only, with no reverse and no genuine round-trip layout pivot (see odfToPdf's own module comment on why pdf -> odf is not attempted) -- so it is modelled as false here even though its own one-way edge is still present as a special case in local.ts. readonly hasLayoutPath: boolean; + // Whether this format can only be READ. A read-only format is a legitimate conversion SOURCE and can never be a target: its codec ships a reader and no writer, so there is nothing to build its bytes from. Stated per format rather than inferred, and required on every entry so a new format has to answer it -- the composition engine, buildDocumentBytes, and setDocumentMetadata each enforce the same answer at their own boundary, and READ_ONLY_FORMATS below is the one derivation they share. + readonly readOnly: boolean; } export const FORMAT_CAPABILITIES: Readonly< Record > = { - docx: { format: "docx", variant: "wordprocessing", hasLayoutPath: true }, - odt: { format: "odt", variant: "wordprocessing", hasLayoutPath: true }, - pptx: { format: "pptx", variant: "presentation", hasLayoutPath: true }, - odp: { format: "odp", variant: "presentation", hasLayoutPath: true }, - ods: { format: "ods", variant: "spreadsheet", hasLayoutPath: true }, + docx: { + format: "docx", + variant: "wordprocessing", + hasLayoutPath: true, + readOnly: false, + }, + odt: { + format: "odt", + variant: "wordprocessing", + hasLayoutPath: true, + readOnly: false, + }, + pptx: { + format: "pptx", + variant: "presentation", + hasLayoutPath: true, + readOnly: false, + }, + odp: { + format: "odp", + variant: "presentation", + hasLayoutPath: true, + readOnly: false, + }, + ods: { + format: "ods", + variant: "spreadsheet", + hasLayoutPath: true, + readOnly: false, + }, // xlsx shares the spreadsheet ContentDocument variant with ods (readXlsxContent/buildXlsxPackage, both from ooxml.js) but has no layout-engine path of its own -- there is no convertSpreadsheetToLayout-equivalent xlsx entry point, only ods's. hasLayoutPath stays false: the composition engine routes xlsx <-> pdf through the ods bridge + ods's own layout engine rather than being a genuine ContentDocument -> LayoutDocument pipeline of xlsx's own. - xlsx: { format: "xlsx", variant: "spreadsheet", hasLayoutPath: false }, + xlsx: { + format: "xlsx", + variant: "spreadsheet", + hasLayoutPath: false, + readOnly: false, + }, // csv shares the spreadsheet variant with xlsx/ods (readCsvContent/buildCsvText, src/csv/) and follows xlsx's routing exactly: plain text carries no layout of its own, so csv <-> pdf goes through the ods bridge + ods's layout engine. TSV is this same member with { delimiter: '\t' }, not a separate format -- see port.ts's own csv comment. - csv: { format: "csv", variant: "spreadsheet", hasLayoutPath: false }, - odg: { format: "odg", variant: "drawing", hasLayoutPath: true }, + csv: { + format: "csv", + variant: "spreadsheet", + hasLayoutPath: false, + readOnly: false, + }, + odg: { + format: "odg", + variant: "drawing", + hasLayoutPath: true, + readOnly: false, + }, // svg shares the drawing ContentDocument variant with odg (readSvgContent/buildSvgText, src/svg/) and has a genuine layout-engine edge of its own: svgToPdf feeds the drawing ContentDocument it reads straight into the same convertDrawingToLayout engine odgToPdf already uses, so hasLayoutPath is true -- unlike csv's text-only entry, plain SVG text still describes real page geometry (a root viewBox is a page size), and the drawing layout engine renders it. - svg: { format: "svg", variant: "drawing", hasLayoutPath: true }, + svg: { + format: "svg", + variant: "drawing", + hasLayoutPath: true, + readOnly: false, + }, // odf reads into the 'formula' ContentDocument variant (readOdfFormulaContent), but hasLayoutPath stays false: odfToPdf renders its formula through writePdf's own separate formula positioning rather than a ContentDocument -> LayoutDocument layout engine, and there is no reverse pdf -> odf at all (see odfToPdf's own module comment in convert.ts). - odf: { format: "odf", variant: "formula", hasLayoutPath: false }, + odf: { + format: "odf", + variant: "formula", + hasLayoutPath: false, + readOnly: true, + }, // markdown shares the wordprocessing variant with docx/odt (readMarkdownContent produces the identical WordprocessingContentDocument shape -- see convert.ts's own top-of-file comment) and has a genuine layout-engine edge of its own (markdownToPdf/pdfToMarkdown both reuse convertWordprocessingToLayout/reconstructWordprocessing unmodified), unlike xlsx above. markdown: { format: "markdown", variant: "wordprocessing", hasLayoutPath: true, + readOnly: false, }, // rtf shares the wordprocessing variant with docx/odt/markdown (readRtfContent/writeRtfContent, rtf-codec) but follows csv/xlsx's routing exactly rather than markdown's: rtf-codec has no layout engine of its own -- no convertWordprocessingToLayout-equivalent rtf entry point -- so hasLayoutPath stays false and the composition engine routes rtf <-> pdf through a same-variant bridge to docx/odt/markdown plus that format's own layout engine, never a direct rtf -> LayoutDocument pipeline. - rtf: { format: "rtf", variant: "wordprocessing", hasLayoutPath: false }, - pdf: { format: "pdf", hasLayoutPath: false }, + rtf: { + format: "rtf", + variant: "wordprocessing", + hasLayoutPath: false, + readOnly: false, + }, + // wpd shares the wordprocessing variant with docx/odt/markdown/rtf (wpd-codec's readWpdContent), and is the first read-only member: wpd-codec ships a reader and no writer, deliberately -- see that package's own Scope. hasLayoutPath is true for markdown's reason rather than rtf's: what it reads is a wordprocessing ContentDocument convertWordprocessingToLayout renders unmodified, and with no reverse direction to keep symmetrical there is nothing to weigh that against, so wpd -> pdf is a direct layout pass rather than a build-and-re-read through a docx bridge. + wpd: { + format: "wpd", + variant: "wordprocessing", + hasLayoutPath: true, + readOnly: true, + }, + pdf: { format: "pdf", hasLayoutPath: false, readOnly: false }, }; +// Every format that can only be read, derived from the capabilities above rather than restated. This is what buildDocumentBytes and setDocumentMetadata check to refuse a read-only TARGET with a reason instead of an internal-invariant message: a format with no writer is a caller error to name as a target, not a registry gap. The composition engine enforces the same fact structurally instead (nothing points at a read-only node in its graph), so it consults this set for nothing -- one fact, two enforcement points that cannot disagree, because both are downstream of the same declaration. +export const READ_ONLY_FORMATS: ReadonlySet = new Set( + Object.values(FORMAT_CAPABILITIES) + .filter((capability) => capability.readOnly) + .map((capability) => capability.format), +); + // Thrown when a requested (source, target) pair has no route in the composition graph (resolveCompositionPlan returned undefined) -- convertDocument and the local DocumentConverter (local.ts) reject rather than silently routing through a path the engine cannot resolve. A named class matching this package's own OdmUnresolvedSectionError/HsqldbSqlUnsupportedError convention for "recognised but unsupported", so a caller can branch on it rather than string-matching a message. export class UnsupportedConversionError extends Error { readonly source: DocumentFormat; diff --git a/packages/documents.js/src/convert/composition-to-pdf.ts b/packages/documents.js/src/convert/composition-to-pdf.ts index 0182ba83b..f1c06724c 100644 --- a/packages/documents.js/src/convert/composition-to-pdf.ts +++ b/packages/documents.js/src/convert/composition-to-pdf.ts @@ -30,11 +30,13 @@ import { executeBridge, executeFromPdf, FORMAT_NODES, + isReadOnlyContentFormat, isTextFormatNode, LAYOUT_CAPABLE, + READ_ONLY_FORMAT_NODES, resolveCompositionPlan, runCompositionPlan, - type ContentFormat, + type SourceContentFormat, type UnifiedConversionOptions, } from "./composition"; import { UnsupportedConversionError } from "./capability"; @@ -54,7 +56,7 @@ const LAYOUT_ENGINES = { // decode(source) -> [extract source fonts] -> build font registry -> read(source) -> resolve metadata -> layout engine by variant -> writePdf, reproducing the exact sequence and option-threading of convert.ts's *ToPdf functions (docxToPdf/odtToPdf/odpToPdf/odsToPdf/odgToPdf/markdownToPdf). The font registry is built from the source package's own embedded faces for package formats (createDocumentFontRegistry) and from caller-supplied faces alone for markdown (createFontRegistry), matching markdownToPdf's own documented divergence. The drawing engine takes no mathMetricsAt and produces no positioned formulas, so writePdf is called without `formulas` for that variant -- byte-identical to odgToPdf. markdownToPdf's leading throwIfAborted (decodeMarkdownText has no abort hook of its own) is reproduced; the package paths match docxToPdf/odtToPdf by not checking abort until writePdf's own loops do. export function executeToPdf( - format: ContentFormat, + format: SourceContentFormat, bytes: Uint8Array, options?: UnifiedConversionOptions, ): Uint8Array { @@ -63,10 +65,24 @@ export function executeToPdf( `executeToPdf: '${format}' has no layout engine of its own`, ); } - const node = FORMAT_NODES[format]; let content: ContentDocument; let fonts: FontRegistry; + // A read-only source has no package to extract embedded faces from and no decode step to run first (see composition.ts's own ReadOnlyFormatNode), so it takes the caller-supplied-faces-only registry the text formats take -- the identical divergence markdownToPdf's own documentation already names, for the identical reason: there is no source package to ask. + if (isReadOnlyContentFormat(format)) { + throwIfAborted(options?.signal); + const read = READ_ONLY_FORMAT_NODES[format].read(bytes, options); + content = { + ...read, + metadata: resolveMetadataTimestamps(read.metadata, options?.clock), + }; + fonts = createFontRegistry({ + fonts: options?.fonts, + onSubstitution: options?.onFontSubstitution, + }); + return renderToPdf(content, fonts, options); + } + const node = FORMAT_NODES[format]; if (isTextFormatNode(node)) { throwIfAborted(options?.signal); const text = node.decode(bytes); @@ -97,6 +113,15 @@ export function executeToPdf( onFontSubstitution: options?.onFontSubstitution, }); } + return renderToPdf(content, fonts, options); +} + +// The half of executeToPdf that is the same whatever kind of node produced the content: lay the ContentDocument out through its variant's engine, then write the PDF. Split out when read-only sources joined, so the three ways a source's content and font registry are obtained (a package's embedded faces, a text format's caller-supplied ones, a read-only format's caller-supplied ones) each end in one shared render rather than three copies of it. +function renderToPdf( + content: ContentDocument, + fonts: FontRegistry, + options: UnifiedConversionOptions | undefined, +): Uint8Array { const measurer = createFontMeasurer(fonts); // Layout by variant. Every engine returns { document, pages } plus (for the three that render embedded formulas) positioned MathML; the drawing engine takes no mathMetricsAt and produces no positioned formulas, so writePdf omits `formulas` for it -- the exact odgToPdf divergence. Each engine also stamps the placements it computed onto `content`'s own nodes in place (frames), so the content reported below is the fused unified package half, not the bare read output. diff --git a/packages/documents.js/src/convert/composition.ts b/packages/documents.js/src/convert/composition.ts index 0b58e2911..c6dd6e6bd 100644 --- a/packages/documents.js/src/convert/composition.ts +++ b/packages/documents.js/src/convert/composition.ts @@ -53,6 +53,7 @@ import { decodeSvgText, encodeSvgText } from "../svg/text"; import { readSvgContent } from "../svg/read"; import { buildSvgText } from "../svg/write"; import { readRtfContent, rtfBytesFromLatin1, writeRtfContent } from "rtf-codec"; +import { readWpdContent } from "wpd-codec"; import { requireArrayBufferBytes } from "../model/bytes"; import type { CellTypeInferenceSink } from "../layout/cell-typing"; import { @@ -105,7 +106,7 @@ export interface UnifiedConversionOptions { // --- Registry: declarative per-format primitive wiring ----------------------------------------- -// The eleven content formats this engine routes between (pdf is the layout pivot, reached via toPdf/fromPdf edges; odf is special, excluded entirely -- see the module doc). +// The eleven read-and-write content formats this engine routes between (pdf is the layout pivot, reached via toPdf/fromPdf edges; odf is special, excluded entirely -- see the module doc). A format here can be either end of a conversion, which is what the read/build pair in its FORMAT_NODES entry means; the read-only formats that can only ever be a SOURCE are ReadOnlyContentFormat below. export type ContentFormat = | "docx" | "pptx" @@ -180,6 +181,67 @@ export function isTextFormatNode(node: FormatNode): node is TextFormatNode { return !node.hasSourcePackage; } +// --- Read-only formats: a source that can never be a target ------------------------------------- +// +// Some formats in this family have a genuine, tested reader and no writer at all -- not "no writer yet" as an omission, but as a deliberate scope decision, because a half-correct writer for a format nobody can round-trip against is worse than none (wpd-codec's own Scope section states exactly that for WordPerfect). Such a format is a real conversion SOURCE and can never be a target, and that asymmetry is the thing this engine had no way to express: FORMAT_NODES' read/build pair says a format does both, and the graph builder's edges are all bidirectional. +// +// A read-only format is therefore a second kind of node with its own registry and its own DIRECTED edges. Nothing points at one, so the pathfinder can never route TO a read-only format: reachability does the work, and no target-side guard is needed anywhere. What makes it generic rather than a wpd-shaped hole is that adding another read-only format (doc-codec, ppt-codec, and xls-codec are the ones this workspace already has, each unwired for the identical reason) is one union member plus one registry entry -- the graph builder, the executors, and the plan runner all already handle the whole set. +// +// odf is deliberately NOT modelled this way even though it too has a reader and no writer. It reads into the 'formula' variant, which has no layout engine, no reconstructor, and no second format to bridge to -- so a read-only odf node would have zero outgoing edges and route nothing. Its one real edge, odf -> pdf, goes through src/mathml's own formula positioning rather than any executor here, which is why it stays local.ts's special case (see this module's own top comment). +export type ReadOnlyContentFormat = "wpd"; + +// The explicit, typed list, kept in sync with READ_ONLY_FORMAT_NODES' own keys for the same reason CONTENT_FORMATS is: iterating `Object.keys` would return `string[]` and need a cast back. +const READ_ONLY_CONTENT_FORMATS: readonly ReadOnlyContentFormat[] = ["wpd"]; + +// A source-only node. Deliberately not a third member of the FormatNode union: it has no build/encode half at all, so widening that union would make every executor's target-side call site branch on a case that can never occur there, and the "no decode step" shape below would have to be faked with an identity decode. `read` takes bytes directly because a read-only codec has no round trip to keep symmetrical -- there is no encode to be the inverse of a decode, so the intermediate representation that split exists for (a Package, a text string) has nothing to hold. +export interface ReadOnlyFormatNode { + readonly variant: LayoutVariant; + readonly read: ( + bytes: Uint8Array, + options?: UnifiedConversionOptions, + ) => ContentDocument; +} + +// wpd reads WordPerfect 6.x-X6 into the wordprocessing ContentDocument variant (wpd-codec's readWpdContent), so it bridges to docx/odt/markdown/rtf at cost 1 and rides the wordprocessing layout engine to pdf directly -- markdown's own justification for a layout path, since convertWordprocessingToLayout consumes what it reads unmodified. read passes only signal through; readWpdContent's own ReadWpdOptions carries a WpdDiagnosticSink too, but UnifiedConversionOptions declares no field for it, matching csv/svg/rtf's own precedent of surfacing only the options this shared shape already has room for. +export const READ_ONLY_FORMAT_NODES: Readonly< + Record +> = { + wpd: { + variant: "wordprocessing", + read: (bytes) => readWpdContent(bytes), + }, +}; + +// Every format this engine can read FROM: the read-and-write ones plus the read-only ones. This is the type a bridge's or a toPdf hop's SOURCE is, where its target stays the narrower ContentFormat. +export type SourceContentFormat = ContentFormat | ReadOnlyContentFormat; + +export function isReadOnlyContentFormat( + format: DocumentFormat, +): format is ReadOnlyContentFormat { + return (READ_ONLY_CONTENT_FORMATS as readonly DocumentFormat[]).includes( + format, + ); +} + +// Decodes and reads whichever kind of source node this format has, so an executor states the "get a ContentDocument out of these bytes" step once rather than branching on node kind at every call site. The read-and-write half keeps its decode/read split (the package or text intermediate its own encode is the inverse of); the read-only half has none. +function readSourceContent( + format: SourceContentFormat, + bytes: Uint8Array, + options: UnifiedConversionOptions | undefined, +): { readonly content: ContentDocument; readonly variant: LayoutVariant } { + if (isReadOnlyContentFormat(format)) { + const node = READ_ONLY_FORMAT_NODES[format]; + return { content: node.read(bytes, options), variant: node.variant }; + } + const node = FORMAT_NODES[format]; + if (isTextFormatNode(node)) { + const text = node.decode(bytes); + return { content: node.read(text, options), variant: node.variant }; + } + const pkg = node.decode(bytes); + return { content: node.read(pkg, options), variant: node.variant }; +} + // The bytes -> latin1-string half of the round trip FORMAT_NODES.rtf's decode/build use (see TextFormatNode's own comment on why this exists): each byte becomes exactly one UTF-16 code unit 0x00-0xFF, the inverse of rtf-codec's own rtfBytesFromLatin1. Chunked at 8192 bytes per String.fromCharCode call, mirroring rtf-codec's own internal asciiStringFromBytes (src/bytes.ts, not part of that package's public surface) -- spreading an unbounded byte array as call arguments in one shot risks "Maximum call stack size exceeded" well before a real .rtf file's own DEFAULT_MAX_INPUT_BYTES (64 MiB, mostly hex-encoded picture payload) is reached. const LATIN1_CHUNK_SIZE = 8192; function rtfBytesToLatin1(bytes: Uint8Array): string { @@ -325,8 +387,10 @@ export const FORMAT_NODES: Readonly> = { }; // The formats that have a direct layout-engine path to/from PDF (convertXToLayout + writePdf). xlsx and csv are deliberately absent: neither has a layout engine of its own, so the pathfinder routes each <-> pdf through ods instead (e.g. csv -> ods bridge, then ods -> pdf toPdf), reproducing the composed route xlsxToPdf/pdfToXlsx already hard-code in convert.ts. rtf is absent for the identical reason, routed through a same-variant bridge to docx/odt/markdown instead. svg is present: its read half produces a drawing ContentDocument whose page geometry comes from the svg root's own viewBox/width/height, and convertDrawingToLayout renders it unmodified. Exported because composition-to-pdf.ts's executeToPdf is the executor that enforces it. -export const LAYOUT_CAPABLE: ReadonlySet = - new Set([ +// +// wpd is present, and being read-only is exactly why. For a read-and-write format the entry is a judgement between two working routes -- rtf reaches pdf through a docx bridge at a cost the hand-written rtfToPdf already accepted -- but for a read-only one there is no reverse direction to keep symmetrical, and the only question left is markdown's own: does its read produce a ContentDocument the variant's layout engine consumes unmodified? readWpdContent produces a wordprocessing document that convertWordprocessingToLayout renders exactly as it renders docx's, so routing wpd -> pdf through a docx bridge instead would build and re-read an OOXML package for nothing, losing whatever that builder cannot express on the way through. +export const LAYOUT_CAPABLE: ReadonlySet = + new Set([ "docx", "pptx", "odt", @@ -335,6 +399,7 @@ export const LAYOUT_CAPABLE: ReadonlySet = "odg", "svg", "markdown", + "wpd", ]); // Cross-variant transforms keyed by `${fromVariant}->${toVariant}`. Each wrapper narrows its input with a runtime kind guard so the underlying transform receives its exact concrete variant type -- the same "no cast, narrow at the boundary" discipline every read/build closure above follows. Today wordprocessing <-> presentation and drawing <-> presentation transforms exist (src/convert/variant-bridges.ts); the pathfinder derives its cross-variant edges from this object's keys, so adding a transform here is the single change needed to teach both the pathfinder and the bridge executor a new variant crossing. @@ -392,34 +457,30 @@ const RECONSTRUCTORS: Readonly< // decode(source) -> read(source) -> [optional cross-variant transform] -> build(target) -> encode(target), reproducing the exact sequence and option-threading of convert.ts's bridge functions (odtToDocx/docxToOdt/markdownToDocx/docxToPptx). onMathDiagnostic reaches the docx reader and builder only (via the registry closures); images reach the markdown reader only; throwIfAborted frames the read and build stages exactly as the hand-written bridges do. The pathfinder only proposes a bridge hop when source and target either share a variant (same-variant direct copy) or have a TRANSFORMS entry between their variants (cross-variant semantic transform), so a missing transform here is a pathfinder bug, not a runtime hazard. export function executeBridge( - source: ContentFormat, + source: SourceContentFormat, target: ContentFormat, bytes: Uint8Array, options?: UnifiedConversionOptions, ): Uint8Array { throwIfAborted(options?.signal); - const sourceNode = FORMAT_NODES[source]; const targetNode = FORMAT_NODES[target]; - // Decode + read the source, branching on hasSourcePackage so the package (SourcePackage) and text (string) decoded shapes stay concrete. - let content: ContentDocument; - if (isTextFormatNode(sourceNode)) { - const text = sourceNode.decode(bytes); - content = sourceNode.read(text, options); - } else { - const pkg = sourceNode.decode(bytes); - content = sourceNode.read(pkg, options); - } - if (content.kind !== sourceNode.variant) { + // Read the source through whichever kind of node it has -- readSourceContent states the read-and-write half's decode/read split and the read-only half's bytes-straight-to-content shape once, so a bridge's SOURCE may be either while its target stays a read-and-write format by type. + const { content, variant: sourceVariant } = readSourceContent( + source, + bytes, + options, + ); + if (content.kind !== sourceVariant) { throw new Error( - `executeBridge: ${source} read returned a non-${sourceNode.variant} ContentDocument`, + `executeBridge: ${source} read returned a non-${sourceVariant} ContentDocument`, ); } // Cross-variant bridges apply the semantic transform between read and build (docx -> pptx, odt -> odp, ...). Same-variant bridges copy the content straight through. let buildContent: ContentDocument = content; - if (sourceNode.variant !== targetNode.variant) { - const key = `${sourceNode.variant}->${targetNode.variant}`; + if (sourceVariant !== targetNode.variant) { + const key = `${sourceVariant}->${targetNode.variant}`; const transform = TRANSFORMS[key]; if (transform === undefined) { throw new Error(`executeBridge: no transform registered for ${key}`); @@ -564,6 +625,22 @@ function buildCompositionGraph(): ReadonlyMap< } } + // Read-only formats (see ReadOnlyContentFormat above) get the same three edge kinds at the same three costs, but DIRECTED -- out of the read-only node only. That single asymmetry is the whole mechanism: with nothing pointing at one, Dijkstra can never reach a read-only format as a target, so "a source that can never be a target" is a property of the graph's shape rather than a rule some resolver has to remember to apply. + for (const source of READ_ONLY_CONTENT_FORMATS) { + const variant = READ_ONLY_FORMAT_NODES[source].variant; + for (const target of CONTENT_FORMATS) { + const targetVariant = FORMAT_NODES[target].variant; + if (targetVariant === variant) { + addDirected(source, target, 1); + } else if (TRANSFORMS[`${variant}->${targetVariant}`] !== undefined) { + addDirected(source, target, 2); + } + } + if (LAYOUT_CAPABLE.has(source)) { + addDirected(source, "pdf", 3); + } + } + return adj; } @@ -645,15 +722,24 @@ export function resolveCompositionPlan( return { hops }; } -// Narrows a DocumentFormat to the ContentFormat union (the eleven formats with a FORMAT_NODES entry). pdf and odf are excluded: pdf is the layout pivot reached only via toPdf/fromPdf edges, and odf is the special-case format this engine does not route at all. Used by runCompositionPlan to narrow a hop's DocumentFormat endpoints to the ContentFormat the executors are typed against. +// Narrows a DocumentFormat to the ContentFormat union (the eleven formats with a FORMAT_NODES entry) -- the type every hop's TARGET must be, since a target is built and encoded. pdf, odf, and every read-only format are excluded: pdf is the layout pivot reached only via toPdf/fromPdf edges, odf is the special-case format this engine does not route at all, and a read-only format has no build half to be a target with. function isContentFormat(format: DocumentFormat): format is ContentFormat { - return format !== "pdf" && format !== "odf"; + return ( + format !== "pdf" && format !== "odf" && !isReadOnlyContentFormat(format) + ); +} + +// The same narrowing for a hop's SOURCE, which may additionally be a read-only format. Used by runCompositionPlan for the `from` endpoint of a bridge or toPdf hop, where isContentFormat covers the `to`. +function isSourceContentFormat( + format: DocumentFormat, +): format is SourceContentFormat { + return isContentFormat(format) || isReadOnlyContentFormat(format); } // The executor binding a plan runner dispatches through. bridge and fromPdf are always present (both live in this module); toPdf is bound only by composition-to-pdf.ts's full convertDocument, because the executor that renders a PDF is exactly the half of the engine a read-only caller must not reach. A plan needing a toPdf hop against a binding that carries none fails loudly below -- for the read-only entry that state is unreachable by construction (pdf as a source never routes back through pdf; Dijkstra never revisits a node), so the throw is an internal-invariant guard, not a caller-facing branch. export interface CompositionExecutorBinding { readonly bridge: ( - source: ContentFormat, + source: SourceContentFormat, target: ContentFormat, bytes: Uint8Array, options?: UnifiedConversionOptions, @@ -664,7 +750,7 @@ export interface CompositionExecutorBinding { options?: UnifiedConversionOptions, ) => Uint8Array; readonly toPdf?: ( - source: ContentFormat, + source: SourceContentFormat, bytes: Uint8Array, options?: UnifiedConversionOptions, ) => Uint8Array; @@ -693,7 +779,7 @@ export function runCompositionPlan( ? undefined : { ...options, onDocument: undefined }; if (hop.executor === "toPdf") { - if (!isContentFormat(hop.from)) { + if (!isSourceContentFormat(hop.from)) { throw new Error( `runCompositionPlan: toPdf source '${hop.from}' is not a content format`, ); @@ -713,7 +799,7 @@ export function runCompositionPlan( } current = executors.fromPdf(hop.to, current, hopOptions); } else { - if (!isContentFormat(hop.from) || !isContentFormat(hop.to)) { + if (!isSourceContentFormat(hop.from) || !isContentFormat(hop.to)) { throw new Error( `runCompositionPlan: bridge endpoints '${hop.from}' -> '${hop.to}' are not both content formats`, ); diff --git a/packages/documents.js/src/convert/from-package.ts b/packages/documents.js/src/convert/from-package.ts index 5cec2a20f..6cfc0f49f 100644 --- a/packages/documents.js/src/convert/from-package.ts +++ b/packages/documents.js/src/convert/from-package.ts @@ -28,6 +28,7 @@ import { } from "../layout/shared"; import { DOCUMENT_FORMAT_CODECS } from "../codecs/registry"; import { requireArrayBufferBytes } from "../model/bytes"; +import { READ_ONLY_FORMATS } from "./capability"; import type { DocumentFormat } from "./port"; import type { LayoutDocument, @@ -52,15 +53,16 @@ export function buildDocumentBytes( } return writePdf(layoutDocumentFromPackage(pkg)); } - if (target === "odf") { + // A read-only format (capability.ts's READ_ONLY_FORMATS) has no writer at all, so naming one as a target is a caller error with a real answer rather than a registry gap: 'odf' (a standalone formula document) has no ContentDocument-to-formula path anywhere in the family, and 'wpd' has none because wpd-codec deliberately ships no writer. One check covers both, and covers whichever read-only format joins them next. + if (READ_ONLY_FORMATS.has(target)) { throw new Error( - "'odf' (a standalone formula document) cannot be built from a DocumentTree -- there is no ContentDocument-to-odf builder", + `'${target}' is a read-only format: it cannot be built from a DocumentTree, because there is no ContentDocument-to-${target} builder`, ); } const content = DOCUMENT_FORMAT_CODECS[target].content; if (!content?.write) { throw new Error( - `DocumentFormat '${target}' has no content.write codec in DOCUMENT_FORMAT_CODECS, and is not 'pdf'/'odf' -- this is an internal invariant violation, not a caller error`, + `DocumentFormat '${target}' has no content.write codec in DOCUMENT_FORMAT_CODECS, is not 'pdf', and is not read-only -- this is an internal invariant violation, not a caller error`, ); } return requireArrayBufferBytes(content.write(flattenTree(pkg))); diff --git a/packages/documents.js/src/convert/local.test.ts b/packages/documents.js/src/convert/local.test.ts index 0bfbc7a4a..0215bbf5f 100644 --- a/packages/documents.js/src/convert/local.test.ts +++ b/packages/documents.js/src/convert/local.test.ts @@ -39,7 +39,7 @@ describe("createLocalDocumentConverter: shape", () => { const converter = createLocalDocumentConverter(); // 7, not 6: ConversionResult.package changed TYPE to the tree-form DocumentTree of document-schema.js 4.0.0 (children carry the decomposed group tree plus the minted styles table, where it previously carried the flat { content, pages } envelope) -- see port.ts's own contractVersion comment on what does and does not warrant a bump. expect(converter.contractVersion).toBe(7); - // SUPPORTED_CONVERSIONS is now derived from the composition pathfinder (resolveCompositionPlan) rather than a hand-maintained DIRECT_EDGES list. The pathfinder routes every pair of non-odf formats within the 3-hop cap, plus the special-case odf -> pdf pair -- 129 pairs total, sorted by source then target for determinism. csv joins as a full spreadsheet-variant member: same-variant bridges to ods/xlsx directly, everything else composed through the identical ods pivot xlsx uses. svg joins as the drawing family's plain-text member the same way: a same-variant bridge to odg directly plus its own pdf layout pair, everything else composed through those two edges. rtf joins the wordprocessing family the same way again -- same-variant bridges to docx/odt/markdown, everything else composed through those plus each target's own pdf layout pair -- except rtf<->csv and rtf<->xlsx, the one pair family the pathfinder genuinely cannot route: reaching either needs rtf -> {docx|odt|markdown} (bridge) -> pdf (toPdf) -> ods (fromPdf) -> {csv|xlsx} (bridge), four hops, one more than resolveCompositionPlan's own cap allows (the bound stated on that function: "the most any real route needs... xlsx -> markdown... three hops"). This is the pathfinder correctly reporting "unsupported" for a genuinely-too-indirect pair, not a gap in this wiring -- rtf has no toPdf/fromPdf edge of its own (capability.ts's own FORMAT_CAPABILITIES.rtf) the way markdown does, so it costs one more hop than markdown needs for the identical csv/xlsx pairs. + // SUPPORTED_CONVERSIONS is now derived from the composition pathfinder (resolveCompositionPlan) rather than a hand-maintained DIRECT_EDGES list. The pathfinder routes every pair of non-odf formats within the 3-hop cap, plus the special-case odf -> pdf pair -- 141 pairs total, sorted by source then target for determinism. csv joins as a full spreadsheet-variant member: same-variant bridges to ods/xlsx directly, everything else composed through the identical ods pivot xlsx uses. svg joins as the drawing family's plain-text member the same way: a same-variant bridge to odg directly plus its own pdf layout pair, everything else composed through those two edges. rtf joins the wordprocessing family the same way again -- same-variant bridges to docx/odt/markdown, everything else composed through those plus each target's own pdf layout pair -- except rtf<->csv and rtf<->xlsx, the one pair family the pathfinder genuinely cannot route: reaching either needs rtf -> {docx|odt|markdown} (bridge) -> pdf (toPdf) -> ods (fromPdf) -> {csv|xlsx} (bridge), four hops, one more than resolveCompositionPlan's own cap allows (the bound stated on that function: "the most any real route needs... xlsx -> markdown... three hops"). This is the pathfinder correctly reporting "unsupported" for a genuinely-too-indirect pair, not a gap in this wiring -- rtf has no toPdf/fromPdf edge of its own (capability.ts's own FORMAT_CAPABILITIES.rtf) the way markdown does, so it costs one more hop than markdown needs for the identical csv/xlsx pairs. wpd joins as the wordprocessing family's read-only member (see composition.ts's own ReadOnlyContentFormat): a directed edge to every other member of this table at the identical cost a read-and-write wordprocessing member would carry, but only ever as a source -- there is no reverse direction to list, since wpd-codec ships no writer at all. expect(converter.conversions).toEqual([ { source: "csv", target: "docx" }, { source: "csv", target: "markdown" }, @@ -160,6 +160,18 @@ describe("createLocalDocumentConverter: shape", () => { { source: "svg", target: "pptx" }, { source: "svg", target: "rtf" }, { source: "svg", target: "xlsx" }, + { source: "wpd", target: "csv" }, + { source: "wpd", target: "docx" }, + { source: "wpd", target: "markdown" }, + { source: "wpd", target: "odg" }, + { source: "wpd", target: "odp" }, + { source: "wpd", target: "ods" }, + { source: "wpd", target: "odt" }, + { source: "wpd", target: "pdf" }, + { source: "wpd", target: "pptx" }, + { source: "wpd", target: "rtf" }, + { source: "wpd", target: "svg" }, + { source: "wpd", target: "xlsx" }, { source: "xlsx", target: "csv" }, { source: "xlsx", target: "docx" }, { source: "xlsx", target: "markdown" }, diff --git a/packages/documents.js/src/convert/port.test.ts b/packages/documents.js/src/convert/port.test.ts index a8d7e0829..c4dc0c766 100644 --- a/packages/documents.js/src/convert/port.test.ts +++ b/packages/documents.js/src/convert/port.test.ts @@ -16,6 +16,7 @@ describe("DocumentFormatSchema / DOCUMENT_FORMATS", () => { "csv", "markdown", "rtf", + "wpd", "pdf", ]); expect(DOCUMENT_FORMATS).toEqual(DocumentFormatSchema.options); diff --git a/packages/documents.js/src/convert/port.ts b/packages/documents.js/src/convert/port.ts index 1137dd8fc..c76217da0 100644 --- a/packages/documents.js/src/convert/port.ts +++ b/packages/documents.js/src/convert/port.ts @@ -6,7 +6,7 @@ import { z } from "zod"; // The conversion behaviour modelled as a swappable port/contract, not a hard-wired function -- this workspace's standing "portable runtime and storage boundaries" convention, even though the only implementation today (local.ts) is entirely synchronous under the hood. `convert()` itself stays async and takes a mandatory abort signal regardless of that: it's a portability contract for a future non-local adapter (a remote conversion service, say), not a reflection of the local implementation's own synchronicity. -// 'odf' (an ODF formula document) has exactly one direction wired into this port (odf -> pdf, via odfToPdf -- see local.ts) -- unlike every other member here, there is deliberately no pdf -> odf entry: odmToPdf's own README/gotcha explains why that reverse direction is not attempted (recovering structured MathML from rendered glyphs is a categorically different, OCR-adjacent problem, not a geometry-reconstruction one). 'markdown' shares the wordprocessing ContentDocument variant with docx/odt (see capability.ts's own FORMAT_CAPABILITIES.markdown) -- it has a genuine two-way layout-engine edge to/from pdf (markdownToPdf/pdfToMarkdown), plus direct same-variant bridges to docx and odt, exactly like odt already has to docx. 'csv' shares the spreadsheet ContentDocument variant with xlsx/ods -- like xlsx it has no layout engine of its own (the composition engine routes csv <-> pdf through the ods bridge), and TSV is the SAME member with { delimiter: '\t' } rather than a second enum entry, since a delimiter is a parse option, not a different document format. 'svg' shares the drawing ContentDocument variant with odg -- unlike csv it DOES have a layout path of its own (svg -> pdf renders the read drawing ContentDocument through the same convertDrawingToLayout engine odg feeds), plus a same-variant bridge to odg and pdf-composed routes to everything else. 'rtf' shares the wordprocessing ContentDocument variant with docx/odt/markdown (rtf-codec's own readRtfContent/writeRtfContent) but, like csv, has no layout engine of its own -- rtf <-> pdf routes through a same-variant bridge to docx/odt/markdown plus that format's own toPdf/fromPdf edge, never a direct rtf <-> LayoutDocument pipeline. +// 'odf' (an ODF formula document) has exactly one direction wired into this port (odf -> pdf, via odfToPdf -- see local.ts) -- unlike every other member here, there is deliberately no pdf -> odf entry: odmToPdf's own README/gotcha explains why that reverse direction is not attempted (recovering structured MathML from rendered glyphs is a categorically different, OCR-adjacent problem, not a geometry-reconstruction one). 'markdown' shares the wordprocessing ContentDocument variant with docx/odt (see capability.ts's own FORMAT_CAPABILITIES.markdown) -- it has a genuine two-way layout-engine edge to/from pdf (markdownToPdf/pdfToMarkdown), plus direct same-variant bridges to docx and odt, exactly like odt already has to docx. 'csv' shares the spreadsheet ContentDocument variant with xlsx/ods -- like xlsx it has no layout engine of its own (the composition engine routes csv <-> pdf through the ods bridge), and TSV is the SAME member with { delimiter: '\t' } rather than a second enum entry, since a delimiter is a parse option, not a different document format. 'svg' shares the drawing ContentDocument variant with odg -- unlike csv it DOES have a layout path of its own (svg -> pdf renders the read drawing ContentDocument through the same convertDrawingToLayout engine odg feeds), plus a same-variant bridge to odg and pdf-composed routes to everything else. 'rtf' shares the wordprocessing ContentDocument variant with docx/odt/markdown (rtf-codec's own readRtfContent/writeRtfContent) but, like csv, has no layout engine of its own -- rtf <-> pdf routes through a same-variant bridge to docx/odt/markdown plus that format's own toPdf/fromPdf edge, never a direct rtf <-> LayoutDocument pipeline. 'wpd' (WordPerfect 6.x-X6, wpd-codec) is the first READ-ONLY member: it shares the wordprocessing variant too, but wpd-codec ships a reader and no writer at all, deliberately (a lossless round trip through a function-code stream is a much larger job than reading one, and a half-correct writer is worse than none). So every pair naming wpd as a SOURCE is routable and every pair naming it as a TARGET is not -- an asymmetry the composition engine states in its graph rather than in a rule, since a read-only format's edges are directed out of it and nothing points back (see composition.ts's own ReadOnlyContentFormat). A caller must therefore not assume DocumentFormat membership implies both directions; `createLocalDocumentConverter().conversions` is the runtime source of truth for which pairs actually route. // // Zod-first, matching this package's own convention (src/model/bytes.ts and every document-schema.js-sourced union re-exported above): the schema is the source of truth, DocumentFormat is inferred from it rather than hand-written, and DOCUMENT_FORMATS (below) is derived from the same schema rather than a second, independently-typed literal array that could drift out of sync with it. export const DocumentFormatSchema = z.enum([ @@ -22,6 +22,7 @@ export const DocumentFormatSchema = z.enum([ "csv", "markdown", "rtf", + "wpd", "pdf", ]); export type DocumentFormat = z.infer; diff --git a/packages/documents.js/src/convert/roundtrip-matrix.test.ts b/packages/documents.js/src/convert/roundtrip-matrix.test.ts index 1ea1336fb..4f7154670 100644 --- a/packages/documents.js/src/convert/roundtrip-matrix.test.ts +++ b/packages/documents.js/src/convert/roundtrip-matrix.test.ts @@ -28,6 +28,7 @@ import { import { minimalOdtBytes } from "../test-support/odt"; import { minimalDocxBytes } from "../test-support/docx"; import { minimalPptxBytes } from "../test-support/pptx"; +import { minimalWpdBytes } from "../test-support/wpd"; import { csvToOds, docxToMarkdown, @@ -583,6 +584,8 @@ function fixtureBytes(format: DocumentFormat): Uint8Array { ); case "odf": return odfFormulaBytes(FRACTION_FORMULA); + case "wpd": + return minimalWpdBytes(); case "pdf": // Generate a minimal PDF from a docx so the sweep has real PDF bytes for every pdf-sourced pair. return docxToPdf(minimalDocxBytes()); @@ -623,6 +626,9 @@ function isValidOutput( case "odf": // odf has no content.write and is never a sweep target -- ALL_SUPPORTED_PAIRS is derived from the port's own conversions, which excludes every odf-target pair (see local.ts's own odf/pdf special case). Present here only so the switch is exhaustive over every DocumentFormat member, matching fixtureBytes' own coverage. return false; + case "wpd": + // wpd is read-only (READ_ONLY_FORMATS) and is never a sweep target for the identical reason odf above is not -- ALL_SUPPORTED_PAIRS excludes every wpd-target pair since wpd-codec ships no writer. Present here only for the same exhaustiveness reason as odf's own case. + return false; } } diff --git a/packages/documents.js/src/index.ts b/packages/documents.js/src/index.ts index 3515ba933..591d4441b 100644 --- a/packages/documents.js/src/index.ts +++ b/packages/documents.js/src/index.ts @@ -546,6 +546,8 @@ export type { } from "./svg/diagnostics"; // rtf has no documents.js-local wrapper either, for the identical reason xlsx above does not: rtf-codec's own readRtfContent/writeRtfContent already read/write a real wordprocessing ContentDocument directly, with no extra transformation this package would need to layer on (unlike markdown, which runs its own math-lowering pass over what markdown-codec hands back) -- so both are re-exported as-is here, the same read/write pair CONTENT_READERS.rtf (src/codecs/read.ts) and DOCUMENT_FORMAT_CODECS.rtf (src/codecs/registry.ts) already use internally, now reachable without going through either. Unlike markdown/csv/svg, rtf takes bytes directly rather than decoded text -- RTF is byte-oriented, not UTF-8 text (a \binN run can carry arbitrary raw picture bytes) -- so there is no decodeRtfText/encodeRtfText byte<->text boundary to export alongside it. export { readRtfContent, writeRtfContent } from "rtf-codec"; +// wpd has no documents.js-local wrapper for the identical reason rtf above does not: wpd-codec's own readWpdContent already reads a real wordprocessing ContentDocument directly, the same one CONTENT_READERS.wpd (src/codecs/read.ts) and READ_ONLY_FORMAT_NODES.wpd (src/convert/composition.ts) already call internally, now reachable without going through either. There is no writeWpdContent to re-export alongside it -- wpd-codec ships no writer at all, which is exactly what makes wpd a READ_ONLY_FORMATS member rather than a full read/write pair like rtf. +export { readWpdContent } from "wpd-codec"; // The one-way ContentDocument -> Markdown text renderer covering all five ContentDocument kinds, not just 'wordprocessing' -- buildMarkdownText/markdown-codec's writeMarkdownContent above throw MarkdownUnsupportedDocumentKindError for the other four. renderContentDocumentToMarkdown delegates straight to buildMarkdownText for 'wordprocessing' and otherwise flattens slides/sheets/drawing pages/a bare formula into the same ContentBlock vocabulary first, reporting every degrade decision through its own onDiagnostic option (see src/markdown/render.ts's own module comment). export type { MarkdownRenderDiagnostic, diff --git a/packages/documents.js/src/test-support/wpd.ts b/packages/documents.js/src/test-support/wpd.ts new file mode 100644 index 000000000..64196394b --- /dev/null +++ b/packages/documents.js/src/test-support/wpd.ts @@ -0,0 +1,58 @@ +import { WPD_INDEX_RECORD_SIZE } from "wpd-codec/container/prefix"; + +// A minimal but structurally authentic WordPerfect 6.x file: a real 512-byte prefix header, a one-record (index-header-only) index area, no packets, and a document area built from real WPFF function bytes -- the identical minimal shape wpd-codec's own read.test.ts exercises via its test-support/build-wpd.ts (buildWpdFile with no packets). That module is deliberately excluded from wpd-codec's published dist (its own tsdown.config.ts, matching every sibling codec's own test-support exclusion), so it cannot be imported across the package boundary; this is a small, self-contained port of just the two pieces a packet-free fixture needs (the prefix/index layout and the ASCII-to-function-byte text encoding), grounded in the same WPFF Document Structure fields wpd-codec's own container reader parses. wpd-codec has no writer at all (see that package's own Scope), so unlike every other format's fixture here there is no editor API to build this through -- hand-constructing the container bytes is the only option. + +const PREFIX_HEADER_SIZE = 512; +const FILE_ID = [0xff, 0x57, 0x50, 0x43]; +const HARD_END_OF_LINE = 0xcc; + +function putUint16(bytes: Uint8Array, offset: number, value: number): void { + bytes[offset] = value & 0xff; + bytes[offset + 1] = (value >>> 8) & 0xff; +} + +function putUint32(bytes: Uint8Array, offset: number, value: number): void { + putUint16(bytes, offset, value & 0xffff); + putUint16(bytes, offset + 2, (value >>> 16) & 0xffff); +} + +// The ASCII characters of a string as document-area bytes -- every character in the single-byte printable range passes through unchanged except a space, which WordPerfect represents as the Soft Space function (0x80) rather than byte 0x20 (the international shorthand for the sharp s), matching wpd-codec's own test-support/build-wpd.ts. +function documentAreaText(value: string): number[] { + return [...value].map((character) => { + const code = character.charCodeAt(0); + return code === 0x20 ? 0x80 : code; + }); +} + +function buildPacketFreeWpdFile(documentArea: readonly number[]): Uint8Array { + // one record: the index header, no packets. + const documentAreaStart = PREFIX_HEADER_SIZE + WPD_INDEX_RECORD_SIZE; + const fileSize = documentAreaStart + documentArea.length; + + const bytes = new Uint8Array(fileSize); + bytes.set(FILE_ID, 0); + putUint32(bytes, 4, documentAreaStart); + bytes[8] = 1; // product type: WordPerfect + bytes[9] = 0x0a; // file type: WordPerfect document + bytes[10] = 2; // major version: the 6.x-X6 lineage + bytes[11] = 1; // minor version + putUint16(bytes, 12, 0); // not encrypted + putUint16(bytes, 14, PREFIX_HEADER_SIZE); // pointer to the index area + putUint32(bytes, 16, 5); // the documented reserved long at the head of the extended header + putUint32(bytes, 20, fileSize); + + bytes[PREFIX_HEADER_SIZE] = 2; // index header flags + putUint16(bytes, PREFIX_HEADER_SIZE + 2, 1); // record count: the index header alone + + bytes.set(documentArea, documentAreaStart); + return bytes; +} + +// Two paragraphs of real text, separated by a hard end of line, matching this package's own rtf/markdown sweep fixtures in substance (a heading-like first paragraph, a plain second one) -- WordPerfect's function stream carries no heading construct of its own, so both read back as plain paragraphs. +export function minimalWpdBytes(): Uint8Array { + return buildPacketFreeWpdFile([ + ...documentAreaText("Heading"), + HARD_END_OF_LINE, + ...documentAreaText("A paragraph of text."), + ]) as Uint8Array; +} diff --git a/packages/documents/src/rpc/router.ts b/packages/documents/src/rpc/router.ts index 4accec975..1cc74554e 100644 --- a/packages/documents/src/rpc/router.ts +++ b/packages/documents/src/rpc/router.ts @@ -25,6 +25,7 @@ import { readPdf, readRtfContent, readSvgContent, + readWpdContent, readXlsxContent, setDocumentMetadata, } from "documents.js"; @@ -275,7 +276,7 @@ const SanitizedLayoutDocumentSchema = LayoutDocumentSchema.extend({ images: z.record(z.string(), SanitizedLayoutImageAssetSchema), }); -// Reads a ContentDocument directly from bytes, bypassing the conversion engine entirely -- no target build/encode, no PDF layout pass. Every format's standalone content reader is exported from documents.js (xlsx included since documents.js 2.0 -- before that, xlsx had to detour through the xlsx->ods bridge and read .content off the conversion result). markdown, csv, and svg are the plain-text formats: their readers take the decoded string, not a package, so each decodes its bytes up front the way markdown always has. +// Reads a ContentDocument directly from bytes, bypassing the conversion engine entirely -- no target build/encode, no PDF layout pass. Every format's standalone content reader is exported from documents.js (xlsx included since documents.js 2.0 -- before that, xlsx had to detour through the xlsx->ods bridge and read .content off the conversion result). markdown, csv, and svg are the plain-text formats: their readers take the decoded string, not a package, so each decodes its bytes up front the way markdown always has. wpd is the one read-only format (READ_ONLY_FORMATS): it has exactly this one direction to offer -- a preview is a genuine use of it, unlike a conversion target -- so it takes bytes directly, the same shape rtf uses. function readContentForFormat( format: DocumentFormat, bytes: Uint8Array, @@ -285,6 +286,7 @@ function readContentForFormat( if (format === "csv") return readCsvContent(new TextDecoder().decode(bytes)); if (format === "svg") return readSvgContent(new TextDecoder().decode(bytes)); if (format === "rtf") return readRtfContent(bytes).document; + if (format === "wpd") return readWpdContent(bytes); if (format === "pdf") throw new Error("PDF has no standalone content reader"); const pkg = decodeDocumentPackage(format, bytes); switch (format) { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 5abe478e4..4cd2763c3 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -403,6 +403,9 @@ importers: vitest: specifier: ^4.1.10 version: 4.1.11(@types/node@26.2.0)(@vitest/coverage-v8@4.1.11)(jsdom@30.0.1)(vite@8.2.1(@types/node@26.2.0)(esbuild@0.28.2)(jiti@2.7.0)(terser@5.51.2)(yaml@2.9.0)) + wpd-codec: + specifier: ^1.0.1 + version: link:../wpd-codec packages/document-outline.js: dependencies: @@ -655,6 +658,9 @@ importers: temml: specifier: 0.13.4 version: 0.13.4 + wpd-codec: + specifier: workspace:^ + version: link:../wpd-codec zod: specifier: ^4.4.3 version: 4.4.3