Skip to content

Add opt-in low-memory XLSX conversion for .NET - #207

Merged
shps951023 merged 1 commit into
mainfrom
feat/dotnet-xlsx-low-memory
Nov 8, 2026
Merged

shps951023 merged 1 commit into
mainfrom
feat/dotnet-xlsx-low-memory

Conversation

@shps951023

@shps951023 shps951023 commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Summary

Add opt-in .NET XLSX low-memory conversion for large, simple tables without changing the default conversion path.

  • Add MiniPdfConversionOptions.LowMemory and the CLI --low-memory flag.
  • Stream worksheet XML to temporary storage, keep rows and completed pages on disk, and build PDF content one page at a time.
  • Reuse existing layout, styles, number formatting, Unicode/font embedding and pagination logic.
  • Spool non-seekable inputs to disk and clean up temporary files on success and failure.
  • Explicitly reject unsupported features rather than silently changing rendering or falling back to high-memory conversion.
  • Document the compatibility scope and add an isolated-process memory regression benchmark.

The initial scope excludes formulas, merged cells, conditional formatting, drawings/charts/images, Excel tables, hyperlinks, headers/footers, worksheet extensions, custom/hidden row heights, vertical centering, print areas/titles and fit-to-height pagination. MaxRows, MaxColumns and RowsPerPage are also unsupported in this mode. Worksheet restrictions apply to the entire workbook. Existing behavior is unchanged when LowMemory is omitted.

Memory is bounded by the decoded row/page buffers plus offset indexes, workbook/style metadata and font resources; this is disk-backed conversion, not constant-memory or disk-free streaming. Byte-array overloads still allocate the complete output.

Validation

Validated on Windows using .NET SDK 10.0.103:

dotnet build src\MiniPdf\MiniPdf.csproj --configuration Release --no-restore

Passed for netstandard2.0, net462, net6.0, net8.0 and net9.0; existing warnings remain.

dotnet test tests\MiniPdf.Tests\MiniPdf.Tests.csproj --configuration Release --no-restore --logger "trx;LogFileName=low-memory-full.trx" --results-directory artifacts\xlsx-low-memory

Passed: 437 tests, zero failures and zero skips. Coverage includes output equivalence, Unicode, styles, wrapping, pagination, stream ownership, unsupported features, row/column limits, output failures and temporary-file cleanup.

.\scripts\Measure-DotNet-XlsxMemory.ps1 -RowCounts 20000,60000

Passed: identical PDF SHA-256 hashes and at least 30% peak working-set reduction on the largest case.

Rows / columns Default peak working set Low-memory peak working set Default / low-memory time Pages in both modes
20,000 / 8 174.21 MiB 62.14 MiB 4.288 / 4.849 s 870
60,000 / 8 377.22 MiB 64.84 MiB 8.002 / 10.858 s 2,610

The largest case reduced peak working set by 82.8%. These measurements are not a guarantee for every workbook.

.\scripts\Run-DotNet-VisualBenchmark.ps1 -Suite classic -Format xlsx -ArtifactRoot artifacts\xlsx-low-memory\visual-default -SkipBuild -MinimumScore 0

Completed the full 191-case classic XLSX benchmark with all 191 candidate PDFs regenerated, no missing Microsoft 365 or LibreOffice reference PDFs, and all 1,538 expected image references verified. Overall average was 0.9578, unchanged from the preserved report; no case score or page count changed. The threshold was explicitly set to zero to retain known pre-existing low-scoring cases, not to claim that every case meets 0.95.

Additional isolated CLI checks across all 191 fixtures verified:

  • All 191 default PDFs are byte-identical to a freshly built pristine main at 3fa32683.
  • All 108 supported low-memory fixtures produce byte-identical, valid PDFs.
  • All 83 unsupported fixtures report explicit errors without creating output files.
git diff --check

Passed.

Generated validation outputs are excluded from the commit and retained locally under artifacts/xlsx-low-memory; they have not been uploaded as PR attachments.

Checklist

  • The change is focused and does not include unrelated formatting or generated files.
  • New or changed behavior has automated test coverage where practical.
  • Relevant .NET and/or Rust checks pass.
  • Public API and compatibility effects are documented.
  • Security implications and untrusted-input behavior were considered.
  • I have the right to submit every added source file, fixture, font, image, and other asset under its stated license.
  • Third-party and generated material includes its source and license.
  • Test documents contain no confidential information or personal data.
  • User-facing documentation is updated, including translated README files when the English README changed.
  • Large benchmark outputs are attached or stored as CI artifacts unless they are required canonical evidence.

Summary by CodeRabbit

  • New Features
    • Added an opt-in low-memory mode for converting supported XLSX spreadsheets to PDF, available through the command-line tool and library.
    • The mode uses temporary disk storage and requires writable temporary storage. It supports simple tables; unsupported workbook features and options return an error rather than switching to standard mode.
  • Documentation
    • Added setup guidance, usage examples, compatibility details, and a memory benchmarking script.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 8, 2026 15:08
@coderabbitai

coderabbitai Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Explain Change →

Caution

Review failed

The pull request is closed.

Walkthrough

Adds opt-in low-memory XLSX-to-PDF conversion. The mode stores workbook rows and PDF pages on disk, rejects unsupported workbook features and options, and includes CLI and library entry points, tests, documentation, and a memory benchmark.

Changes: Low-memory XLSX conversion

Layer / File(s) Summary
Expose and route the conversion option
src/MiniPdf.Cli/Program.cs, src/MiniPdf/MiniPdf.cs, documents/README.nuget.cli.md, documents/README.nuget.md
The CLI and library expose the low-memory option. Non-seekable input uses temporary file-backed storage in this mode. The documentation describes the option, its constraints, and temporary-storage requirements.
Prepare workbook data and read rows
src/MiniPdf/LowMemoryWorkbook.cs, src/MiniPdf/ExcelReader.cs
LowMemoryWorkbook stores shared strings and worksheet rows on disk while copying the workbook package. It rejects unsupported workbook and worksheet features. ExcelReader reads the stored rows and uses list-based row interfaces.
Store and write PDF pages on disk
src/MiniPdf/FileBackedList.cs, src/MiniPdf/PdfDocument.cs, src/MiniPdf/ExcelToPdfConverter.cs, src/MiniPdf/PdfWriter.cs, src/MiniPdf/MiniPdf.cs
PdfDocument uses file-backed page storage in low-memory mode. Conversion handles disposal and clears row-position references. PdfWriter builds each page content stream during output.
Verify conversion and measure memory
tests/MiniPdf.Tests/LowMemoryXlsxTests.cs, scripts/Measure-DotNet-XlsxMemory.ps1, documents/README.nuget.md
Tests compare low-memory and default PDF output, check unsupported cases and cleanup, and verify output failures. The benchmark records conversion metrics, compares PDF hashes, and checks the peak working-set threshold.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant ExcelToPdfConverter
  participant LowMemoryWorkbook
  participant ExcelReader
  participant PdfDocument
  participant PdfWriter
  CLI->>ExcelToPdfConverter: Convert with LowMemory enabled
  ExcelToPdfConverter->>LowMemoryWorkbook: Prepare workbook package and rows
  ExcelToPdfConverter->>ExcelReader: Read sheets using stored rows
  ExcelToPdfConverter->>PdfDocument: Create low-memory document and add pages
  PdfWriter->>PdfDocument: Read pages for output
  PdfWriter->>PdfWriter: Build each page content stream during writing
Loading
Merge Risk: Moderate · up to 2b790

The opt-in mode still has confidentiality, compatibility, and large-workbook resource risks. Address private temporary storage, generated Indic text serialization, and excessive processing or retention before merging, alongside the narrower validation defects.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

@shps951023
shps951023 merged commit edeea71 into main Nov 8, 2026
4 of 5 checks passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Temporary files can be created with world-readable permissions on shared Unix hosts, exposing worksheet text and generated page content to other local accounts.

7 open issues
What changed in this PR

Adds opt-in disk-backed XLSX conversion to reduce .NET memory usage while preserving the default conversion path and reusing existing rendering logic.

Changes:

  • Adds the LowMemory API option and CLI --low-memory flag.
  • Stores worksheet rows and PDF pages on disk, generates content page by page, and rejects unsupported features.
  • Adds compatibility documentation, regression tests, and an isolated-process memory benchmark.
File Description
tests/​MiniPdf.Tests/​LowMemoryXlsxTests.cs Tests output equivalence, restrictions, stream handling, and cleanup.
src/​MiniPdf/​PdfWriter.cs Generates low-memory PDF content one page at a time.
src/​MiniPdf/​PdfDocument.cs Adds disposable, disk-backed page storage.
src/​MiniPdf/​MiniPdf.cs Exposes LowMemory and spools non-seekable inputs to disk.
src/​MiniPdf/​LowMemoryWorkbook.cs Extracts worksheet rows and validates unsupported features.
src/​MiniPdf/​FileBackedList.cs Implements temporary storage and row/page serialization.
src/​MiniPdf/​ExcelToPdfConverter.cs Integrates low-memory storage, validation, and resource cleanup.
src/​MiniPdf/​ExcelReader.cs Supports disk-backed rows and shared strings.
src/​MiniPdf.Cli/​Program.cs Adds --low-memory parsing and help text.
scripts/​Measure-DotNet-XlsxMemory.ps1 Compares memory usage, timing, and PDF hashes.
documents/​README.nuget.md Documents API usage, compatibility limits, and memory tradeoffs.
documents/​README.nuget.cli.md Documents the CLI flag and links to limitations.

Review effort: Balanced

Comment on lines +29 to +31
internal static FileStream CreateTemporaryStream()
=> new(Path.Combine(Path.GetTempPath(), "minipdf-" + Guid.NewGuid().ToString("N") + ".tmp"),
FileMode.CreateNew, FileAccess.ReadWrite, FileShare.None, 65536, FileOptions.DeleteOnClose);
Comment on lines +1837 to +1840
storedRows.Add(ReadSheet(entry, sharedStrings, boldPrefixLengths, fontStyles, fillColors, borders,
numberFormats, cellXfFontIndices, cellXfFillIndices, cellXfNumFmtIds, cellXfAlignments,
cellXfVerticalAlignments, cellXfTextRotations, cellXfBorderIndices, cellXfWrapTexts,
cellXfIndents, hyperlinks, culture, worksheet: rowDocument)[0]);
Comment on lines +125 to +128
if (options.LowMemory)
foreach (var sheet in sheets)
if (sheet.FitToPage && sheet.FitToHeight > 0)
LowMemoryWorkbook.Unsupported("fit-to-height pagination");
Comment on lines +56 to +58
if (entry.FullName.StartsWith("xl/worksheets/", StringComparison.Ordinal) &&
entry.FullName.EndsWith(".xml", StringComparison.Ordinal) &&
!entry.FullName.Contains("/_rels/"))
Comment on lines +70 to +71
if (value is "_xlnm.Print_Area" or "_xlnm.Print_Titles")
Unsupported(value);
Comment on lines +133 to +134
if (row.Descendants().Any(element => element.Name.LocalName == "f"))
Unsupported("formulas");
Comment thread src/MiniPdf/PdfWriter.cs
Comment on lines +494 to +497
var contentStreams = new List<byte[]>(document.LowMemory ? 0 : pageCount);
if (!document.LowMemory)
for (var i = 0; i < pageCount; i++)
contentStreams.Add(Compat.Latin1.GetBytes(BuildContentStream(pages[i], embeddedFonts.Count > 0, cpToFontSlot, embeddedFonts, fontNameToSlot)));
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants