Skip to content

feat: read and write portable bytes through RawSpan - #31

Closed
vsarunas wants to merge 1 commit into
RoaringBitmap:masterfrom
ordo-one:rawspan
Closed

vsarunas wants to merge 1 commit into
RoaringBitmap:masterfrom
ordo-one:rawspan

Conversation

@vsarunas

@vsarunas vsarunas commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Hi @lemire !

The PR drops not required mallocs by adopting a modern Swift API.

  • Adds RawSpan versions of the portable format API, so bytes can be read where they are instead of being copied into an array first.
  • init?(portableBytes:) deserializes, returning nil if the bytes don't parse.
  • withPortableView(of:) gives a read-only view over the bytes without copying them.
  • portableSerialize(into:) writes into an OutputRawSpan.
  • RoaringBitmap.portableSerialize() -> [UInt8], which Roaring64Bitmap already had.
  • README section with an example.
  • Both RoaringBitmap and Roaring64Bitmap. Swift 6.2.

New sanitisers workflow:

  • Unit tests under AddressSanitizer on macOS.
  • Fuzzer: libFuzzer + AddressSanitizer on Ubuntu, 120 s per run.

Performance improvement

The old frozen API copies the bytes and allocates a bitmap object; the view does neither. Shown on weather_sept_85, where the gain is largest (~8x); the other datasets gain 1.4-4x.

./fetch-datasets.sh
swift package benchmark --filter 'frozen/portable_deserialize_frozen.*/weather_sept_85'

Before, old portableDeserializeFrozen(bytes:):

frozen/portable_deserialize_frozen_copy/weather_sept_85
╒══════════════════════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╕
│ Metric                   │        p0 │       p25 │       p50 │       p75 │       p90 │       p99 │      p100 │   Samples │
╞══════════════════════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╡
│ Instructions (K) *       │      2594 │      2595 │      2597 │      2597 │      2599 │      2628 │      2691 │      5763 │
├──────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Malloc (total) *         │       600 │       600 │       600 │       600 │       600 │       600 │       600 │      5763 │
├──────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Time (wall clock) (μs) * │       156 │       166 │       168 │       172 │       177 │       194 │       294 │      5763 │
╘══════════════════════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╛

After, new withPortableView(of:):

frozen/portable_deserialize_frozen/weather_sept_85
╒══════════════════════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╕
│ Metric                   │        p0 │       p25 │       p50 │       p75 │       p90 │       p99 │      p100 │   Samples │
╞══════════════════════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╡
│ Instructions (K) *       │       497 │       497 │       498 │       500 │       500 │       502 │       543 │     10000 │
├──────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Malloc (total) *         │       200 │       200 │       200 │       200 │       200 │       200 │       200 │     10000 │
├──────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Time (wall clock) (μs) * │        19 │        20 │        20 │        20 │        20 │        24 │       146 │     10000 │
╘══════════════════════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╛

@vsarunas vsarunas changed the title feat(minor): read and write portable bytes through RawSpan feat: read and write portable bytes through RawSpan Oct 24, 2026
lemire added a commit that referenced this pull request Oct 26, 2026
feat(minor): read and write portable bytes through RawSpan (rebased #31)
@lemire lemire closed this Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants