Read WordPerfect page geometry, tables, styles, outline numbering, and metadata - #864
Merged
Conversation
Mearman
force-pushed
the
feat/wpd-read-depth
branch
from
September 3, 2026 10:48
fd07bde to
fe00d19
Compare
4 tasks
Mearman
marked this pull request as ready for review
September 3, 2026 10:50
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Mearman
force-pushed
the
feat/wpd-read-depth
branch
from
September 3, 2026 10:55
fe00d19 to
f64c15e
Compare
Adds one module per structure the reader could not previously state, each built from the SDK page it cites and tested against byte sequences assembled from that page's own field tables. stream/units holds the WordPerfect Unit, one 1200th of an inch, which every dimension in the format but a font size is stated in. stream/page reads the Form function's page size and the four margin functions, which are split across two groups: the vertical pair is page oriented and the horizontal pair is column oriented, because text columns subdivide the space those margins bound. A landscape form's dimensions go through as written rather than rotated, since the specification states width, length and orientation as three independent fields and never says whether the pair is written before or after the rotation. stream/table holds the Table Column layout and the End-of-Line group's embedded subfunctions, where a cell's spanning, justification, fill and fixed row height actually ride. That region is unlike every other function's: it opens with a size word for a deletable half, and each record is self-sized only through the size the SDK prints against it, so a code with no stated size ends the walk instead of being stepped over by a guessed length. stream/style maps the system style number the Style group's own openers carry: 68 through 75 are heading levels 1 through 8, 52 through 67 are outline levels, and 31 and 48 are lists. Scope pairing comes from the flags byte rather than the subfunction names, which is what makes three openers and three closers out of twelve subfunctions. container/summary reads the Extended Document Summary packet, keying on each field's tag rather than its name, which the SDK says comes from a translatable resource file and so cannot be relied on.
…metadata into the content The fold previously turned every function group but end-of-line, paragraph and character into nothing, so a document read back as US Letter with one-inch margins whatever its form said, a table arrived as a run of paragraphs, and metadata was always an empty envelope. The section now carries the geometry the document states, per dimension: each of the page size and the four margins falls back to the WordPerfect default only when its own function never appears, so a document overriding one margin keeps the default for the other three rather than for none of them. A later change to a dimension already stated is reported rather than applied, since a section carries one geometry. A table becomes a real ContentTable. The fold gains its one nesting concept for it: while a definition is open a closed paragraph joins the cell being built rather than the section's block list. Merged cells become colSpan and rowSpan with the positions they cover dropped, which is how the shared schema states a merge; a cell's own justification lands on the paragraphs it holds, because that is where the schema carries alignment. Table Off closes a cell only when one is still open, so the two spellings a document may use -- ending its last row explicitly or leaving it to the table's end -- both produce the same rows. A paragraph's heading level and outline list level come from the style region enclosing it, captured when the paragraph's first character arrives rather than when it closes: a style region ends at its own closing code, which sits before the hard return that ends the paragraph, so reading the scope at flush time would find it already popped. An outline number's rendered digits are replaced by the list membership that regenerates them, so a converted document numbers the item itself rather than carrying a frozen number as text. Boxes, notes, page furniture, cross-references and merge codes are each reported once through the diagnostic sink, so a document losing one says so instead of passing over it in silence.
…ng gap is one The scope sections described a reader that recovered no page geometry, no table structure and no metadata, all of which it now does. The remaining gaps are restated as reasons rather than a list: a box's prefix IDs are identified by a nested tree of override flags rather than positionally, so the list cannot be read without walking that tree; a header, footer or note's text is genuinely recoverable from its own General WP Text packet but has no position in the flat content model, closing at the schema boundary rather than here; and character sets 2 and above are a missing source rather than unfinished work, since the mirrored SDK states the mechanism but tabulates no set of its own. Adds the two readings the specification does not settle -- a landscape form's dimensions and a table column's untagged width unit -- to the honest-limits section beside the absence of a real corpus.
…together The Tab group is the second most common variable-length function in real WordPerfect documents and the fold turned every one of them into nothing, which ran adjacent columns into one word: a class list read back as "Naba Raj AdhikariNepal", a circular's reference number as "Ref. T2/6.01COMSAR/Circ.15", a numbered clause as "1The Sub-Committee". That is a text loss rather than a formatting one. The group is shaped unlike every other and that is what made it easy to overlook: it has no subfunction catalogue at all. The byte sitting where every other group puts a subfunction number is the tab definition itself, a bitfield whose top five bits name the type, so there is one parameterised function here rather than a list of them. A type that advances to a tab stop becomes the tab character, which is the only spelling the shared content model has for one -- it carries no tab-stop table, and the position an indent moves to comes from the document's own table rather than from the function. Three of the types are not tabs at all: centre-on-margins, centre-on-current-position and flush-right begin a line-scoped alignment, and they are the missing half of a construct this reader already handled the end of, since the single-byte End of Center Align functions are what terminate them. A line-scoped alignment outranks Set Justification Mode for the paragraph it sits in, because the one applies to a line and the other from where it sits onwards.
…at it does not The scope sections claimed no real-world corpus, which is no longer true. Ninety-three real WordPerfect 6.x-X6 documents -- libwpd's own oss-fuzz seed corpus plus the Open Preservation Foundation's format corpus -- now back the read path, and the Evidence section states what that does and does not prove. The character model is the finding worth recording: bytes 1 through 32 being accented-letter shorthands rather than ASCII, so that a space is a function code and byte 0x20 is the sharp s, was the assumption most likely to make every real document read back as nonsense. Real documents read back as prose, which they could not do if it were wrong. Three files are refused and each refusal is correct: a WordPerfect 3 for Macintosh file on its major-version byte, an encrypted document, and one whose gates genuinely do not match -- a bug-report attachment, in a corpus collected precisely because its files broke something. Character sets 2 and above turn from a theoretical gap into a measured one: two thirds of the extended characters in real documents name set 4 alone. Heading recovery moves the other way, since no document in the corpus uses WordPerfect's own heading styles, so that mapping rests on the specification's enumeration alone and the README now says so.
Mearman
force-pushed
the
feat/wpd-read-depth
branch
from
September 3, 2026 11:11
f64c15e to
34d3baa
Compare
Contributor
|
🎉 This PR is included in version 1.1.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Deepens
wpd-codec's read coverage against the five items from the scope comment on #819 that the specification actually settles. Everything is derived from Corel's own SDK help, and every constant and offset cites the page it came from, matching what the package already does.Page geometry (0xD1, 0xD2). The section used to be US Letter with one-inch margins for every document regardless of what it said. It now reads the Form function's own page size and all four margins — the vertical pair from the Page group, the horizontal pair from the Column group, which is where a left or right margin lives in this format. Each dimension falls back independently, so a document overriding one margin keeps the default for the other three rather than for none of them; a later change to a dimension already stated is reported rather than applied, since a
ContentSectioncarries one geometry.Tables (0xD42A/2B/2C plus the End-of-Line boundaries). A table is stated in two halves in two different groups — the definition and its column widths in the Character group, the cell and row boundaries in the End-of-Line group — and neither is nested in the other. Both now read, into a real
ContentTable. The per-cell facts ride inside the End-of-Line function that closes the cell, as embedded subfunctions, sosrc/stream/table.tswalks those: merged cells becomecolSpan/rowSpanwith the covered positions dropped (which is how the shared schema states a merge), plus cell background colour, per-cell justification applied to the paragraphs it holds, and fixed row heights.Styles (0xDD). The system style number the Style group's openers carry is enumerated by the SDK, and its entries include heading levels 1-8 and outline levels 1-8. A paragraph's
headingLeveland list level now come from that, never from a heuristic. Scope pairing comes from the flags byte rather than the subfunction names.Outline numbering (0xDA). A Paragraph Number Display pair becomes a
ContentListMembershipat the level it names, with its rendered digits dropped in favour of the membership that regenerates them.Document metadata (prefix packet 0x12). The Extended Document Summary now populates
metadata— Descriptive Name as the title, plus author, subject, keywords, and the creation and revision dates. Keyed on each field's tag, since the SDK says the name comes from a translatable resource file.What is still a gap, and why
Each is now reported through the diagnostic sink rather than passed over silently, and the README states the reason rather than just listing it:
ContentImageBlock'spng/jpegpair cannot hold.ContentDocumenthas no page-furniture position and no note position. A footnote body's real home isdocument-schema.js's tree-onlydefinitionstable, which a codec producing the flat form cannot reach. Verified as the identical gaprtf-codecdocuments for its own equivalents, not assumed.Honest limits
Still no real
.wpdfile behind any of this — every fixture is assembled from the specification's own field tables. Two readings the specification does not settle are called out in the README: a landscape form's dimensions go through as written rather than rotated (width, length and orientation are three independent fields and nothing says whether the pair is pre- or post-rotation), and a Table Column's[width]is read as WordPerfect Units on the strength of the two gutter fields immediately after it being tagged as such.Test plan
pnpm exec turbo run _lint _typecheck _typecheck:node _test _test:workers _test:smoke --filter=wpd-codec— all greenPart of #819. The composition-engine wiring the issue also asks for is a separate PR stacked on this one.