Skip to content

fix(csv-generate): stream object records independently of column values - #522

Open
lux-liang wants to merge 1 commit into
adaltas:masterfrom
lux-liang:fix/object-mode-record-streaming
Open

lux-liang wants to merge 1 commit into
adaltas:masterfrom
lux-liang:fix/object-mode-record-streaming

Conversation

@lux-liang

Copy link
Copy Markdown
Contributor

csv-generate sizes object records by adding each column's .length, although column functions explicitly support strings, numbers and null. A null column throws, and numeric or empty-string columns generate the entire finite input before emitting the first record. With sleep, buffered records also share the end flag and can be dropped: five numeric records currently return only one.

Emit one object record per read so the Readable stream controls when generation resumes. This also preserves the delay between records and finishes only after the last record is delivered. Text-mode chunk sizing is unchanged.

For example, this callback should receive five [42] records:

generate(
  { objectMode: true, columns: [() => 42], length: 5, sleep: 5 },
  (err, records) => console.log(err, records),
);

The regressions cover null values in the sync API, bounded generation before the first streamed record, supported value preservation, and complete delayed output. Against the original source, the three affected test files produced 7 failures and 9 passes; with the fix, all 16 pass.

Validation on Ubuntu with Node 24.19.0 and the lockfile-installed dependencies:

  • npm ci, including the original Husky setup.
  • npm run build passed.
  • npm test passed for all 14 workspaces: 1,085 passing tests and 6 existing pending tests. The first full attempt hit the existing one-second sleep test's timing assertion; the unchanged full command passed on rerun. No assertion or test selection was relaxed.
  • npm run lint:check passed with no errors and 6 existing pending-test warnings.
  • Normal pre-commit and commit-message hooks passed.

Generated distributions are left to the existing release build process.

AI disclosure: this change and its tests were developed with OpenAI Codex and checked through the public APIs and project test suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant