Skip to content

refactor(mpsc): replace the legacy queue backend - #247

Open
mxsm wants to merge 3 commits into
apache:mainfrom
mxsm:mxsm-209
Open

refactor(mpsc): replace the legacy queue backend#247
mxsm wants to merge 3 commits into
apache:mainfrom
mxsm:mxsm-209

Conversation

@mxsm

@mxsm mxsm commented Aug 30, 2026

Copy link
Copy Markdown
Member

Summary

  • Replace the bounded and unbounded std::sync::mpsc backends with MPSC-owned queue implementations while preserving the public sender, receiver, error, and auto-trait contracts.
  • Use receiver-local batching for the unbounded queue and a preallocated, generation-stamped slot ring for the bounded queue.
  • Keep backpressure, cancellation safety, FIFO ordering, disconnection, buffered-message draining, and exactly-once destruction covered by tests.
  • Remove the receiver types' manual Sync implementations and document the bounded slot's localized unsafe invariant.
  • Update the changelog for the backend replacement.

Closes #209.

Design Notes

The unbounded queue keeps sender-visible storage and receiver liveness behind one mutex, while the single receiver moves messages into a local batch to reduce shared-lock contention without introducing unsafe code.

The bounded queue allocates its capacity up front and uses per-slot generation stamps. Producers reserve slots through the tail cursor, while the single consumer advances the head cursor; cache padding prevents producer and consumer cursor updates from sharing a cache line.

A straightforward safe bounded implementation using one Mutex<VecDeque> passed the existing tests and removed the MPSC unsafe code, but orthur2 measured it at roughly 1.7–1.9x slower with four producers and 2.3–3.0x slower with eight on an Apple M5 MacBook Pro (10-core arm64) running macOS 26.4. The stamped ring is retained to avoid that contention regression.

Unsafe code is confined to initialized slot access and Slot<T>: Sync. Each unsafe operation is paired with a documented reservation, publication, acquisition, and reuse invariant. The queue exposes no references to stored values, completes ownership transitions before user code can unwind, preserves the legacy endpoint auto-trait matrix, and exercises receiver-disconnect cleanup across a wrapped ring under Miri.

Validation

  • cargo x test
  • cargo x check
  • cargo x miri (74 tests, including wrapped receiver-disconnect cleanup)
  • cargo x lint (Clippy, rustfmt, typos, license, and rustdoc pass; local Taplo reports pre-existing formatting differences in three unchanged Cargo.toml files)
  • cargo x bench

Benchmark Comparison

The following bounded MPSC medians were measured in adjacent runs on an Intel Core i7-11700K (8 cores/16 threads) running 64-bit Windows 11 Pro 10.0.26200, with 16,384 messages per sample.

Producers Legacy backend New backend Change
1 2.235 ms 1.567 ms -29.9%
2 1.601 ms 1.541 ms -3.7%
4 1.888 ms 2.095 ms +11.0%
8 2.235 ms 2.292 ms +2.6%

In orthur2's comparison on the Apple M5 machine described above, the new unbounded backend was about 1.6x faster than the base with one producer and 7.3x faster with eight producers.

Producers Unbounded change versus base
1 ~1.6x faster
8 ~7.3x faster

The existing ecosystem benchmark continues to cover bounded and unbounded channels with 1, 2, 4, and 8 producers and 16,384 messages per sample.

@tisonkun
tisonkun requested a review from orthur2 August 30, 2026 17:26
@orthur2

orthur2 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

OK. I'll review this PR tomorrow, or within the next few days.

@tisonkun tisonkun mentioned this pull request Aug 31, 2026
35 tasks

@orthur2 orthur2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on.

For reference, I ran these local benchmarks on an Apple M5 MacBook Pro (10-core CPU and arm64) running macOS 26.4.

I also wanted to check whether the bounded queue really needed the stamped ring. I replaced it locally with a naive single-lock Mutex<VecDeque> queue. That version passed the existing tests and removed all unsafe code from the MPSC module, but it was roughly 1.7–1.9x slower with four producers and 2.3–3.0x slower with eight. A more sophisticated safe implementation might do better, but the straightforward version would clearly regress the existing contention path. It would be useful to capture that tradeoff briefly in the Design Notes.

The description says that the full benchmark suite covers both channel flavors, but the only compare numbers shown are for bounded MPSC. Could we also include the unbounded figures and the machine and OS? On the same machine, the new unbounded backend was about 1.6x faster than the base with one producer and 7.3x faster with eight. Those numbers show the main performance benefit of this change much more clearly than the bounded-only table.

I would be happy to take another look once these comments are addressed.

Comment thread asyncband/src/mpsc/queue.rs
Comment thread asyncband/src/mpsc/queue.rs
Comment thread asyncband/src/mpsc/queue.rs
@mxsm

mxsm commented Sep 2, 2026

Copy link
Copy Markdown
Member Author

@orthur2 All requested changes are addressed in 13a100e: the legacy Unpin/unwind-safe endpoint traits are restored with the 80-assertion regression matrix, the wrapped receiver-disconnect cleanup path now has an exactly-once drop test exercised by Miri, and the PR description includes the bounded safe-queue tradeoff, unbounded comparison, and benchmark environments. The three review threads are resolved; could you take another look when convenient?

@mxsm
mxsm requested a review from orthur2 September 2, 2026 05:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(mpsc): replace the legacy queue backend

3 participants