Skip to content

svac3: a faster entropy decoder - #10

Merged
widgetii merged 2 commits into
masterfrom
svac3-aec
Oct 29, 2026
Merged

widgetii merged 2 commits into
masterfrom
svac3-aec

Conversation

@widgetii

@widgetii widgetii commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

After #8 and #9, high-bitrate streams are bound by entropy decoding, which is serial. This PR makes it cheaper per bin without changing any bin.

Engine

  • New form: the draft keeps the interval as shift counts plus 8-bit remainders (rS1/rT1, valueS/valueT, 5.3.3.2.2). The decoder now keeps a 9-bit range and a 32-bit window of the bitstream, refilled 16 bits at a time with a marker bit under the unread bits. That is the form AVS3 decoders use (cf. uavs3d's lbac_dec_bin).
  • Per bin: a decision is one compare, the draft's sFlag as a 0/1 shift, and one leading-zero count to renormalise.
  • Bypass: the normative definition, a decision with lgPmps 1024 (5.3.3.3.3.4), which reduces to one compare and a 1-bit shift.
  • Stuffing bit: a decision with an LPS range of 1.
  • Models unchanged: the probability models, their MAEC updates and the two-model weighting are untouched.
  • Inline: decisions, two-model decisions and bypass bins are now inline in svac3_aec.h, so the patch parser no longer makes a call per bin.
  • Bookkeeping: bits consumed, used for bytes_used and the overrun/CORRUPT check, are recovered exactly from the marker bit (8 × bytes loaded − 16 + ctz(low)).

Smaller changes

  • SRCC's context history (the last five coefficients' significance and levels > 1 / > 2) is now three 5-bit masks with a popcount table, instead of shifted arrays recounted per coefficient.
  • pos_class() is now per-block row and column zones plus a table.
  • Dequantisation skips the zero coefficients.

Equivalence

  • Reference kept: tests/aec_ref.c keeps the previous, draft-shaped engine.
  • New test_aec: runs both engines over random valid AEC data, with random sequences of decisions on a model pool, two-model decisions, bypass bins and stuffing bits, MAEC on and off, I and P thresholds, and zero runs up to just under the 254-bit bound. Bypass is exercised in both forms: the normative decision and Annex M's shortcut.
  • What must agree: every bin, every model, and the bits consumed at each patch end. Elsewhere the draft engine has read ahead over zeros, so its position isn't comparable. Result: about 29 million bins in 5,000 streams, 3,700+ patch ends.
  • Excluded data: buffers whose first nine bits are all 1. Their initial value isn't below the initial range, so no encoder writes them, and the two engines decode such garbage differently. The test also asserts value < range before every operation.
  • Other checks:
    • ctest 26/26 bit-exact in release and in ASan+UBSan, on x86 and on AArch64;
    • fuzzer clean on the new engine (4 minutes, 6 workers, seeds from four corpus streams).

Speed: one core, best of 5, master against this branch, on two idle machines:

stream Core Ultra 5 245K (AVX2) Cortex-X925 (NEON)
1080p fixed QP 22 ×20 (entropy-bound, 120 pictures) 2.01 → 1.56 s (1.29×) 1.80 → 1.39 s (1.29×)
1080p fixed QP 22 1.25× 1.25×
1080p, 9 patches 1.13× 1.14×
1080p SmartP 1.10× 1.10×
1080p CBR 1.09× 1.07×

On the x86 machine, too: test_dsp and test_aec agree, ctest is 26/26 bit-exact. (The earlier x86 figures came from a shared, loaded host and are replaced by these.)

The arithmetic decoder keeps the interval as a 9-bit range and a 32-bit
window of the bitstream, refilled 16 bits at a time, instead of the
draft's shift counts and 8-bit remainders (rS1/rT1, valueS/valueT): the
form AVS3 decoders use, one comparison per bin and a leading-zero count to
renormalise. The probability models and their updates are unchanged.
Decisions, two-model decisions and bypass bins are inline, so the patch
parser no longer calls out for every bin. Bits consumed (for bytes_used
and the overrun check) come from the window's marker bit.

The two forms give the same bins for any AEC data an encoder can write.
tests/aec_ref.c keeps the draft's engine, and test_aec runs both over
random valid data with random sequences of decisions, two-model
decisions, bypass bins (both the normative definition, a decision with
lgPmps 1024, and Annex M's shortcut) and stuffing bits: every bin and
model agrees, and so do the bits consumed at each patch end. Data whose
first nine bits are all 1, which no encoder writes, is excluded: the
engines part there, as garbage in either.

Also: the coefficient-context history of SRCC (the last five levels) as
bit masks with a popcount, pos_class() as per-block row and column zones,
and the dequantisation skips the zero coefficients.
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Speed up SVAC3 entropy decoding without changing decoded bins

✨ Enhancement 🧪 Tests 🕐 40+ Minutes

Grey Divider

AI Description

• Replace the draft-shaped entropy engine with an inlined, window-based decoder to reduce per-bin
 cost.
• Streamline coefficient contexts and skip zero-coefficient dequantisation work.
• Compare decoded bins, models, and patch-end bit counts against the preserved reference engine.
Diagram

graph TD
  P["Patch parser"] --> W["Bitstream window"] --> A["Arithmetic bins"] --> C["MAEC contexts"] --> S["SRCC coefficients"] --> D["Sparse dequant"]
  W --> B["Patch accounting"]
Loading
High-Level Assessment

Keep the window-based engine and the independent draft-shaped test reference. Merely inlining the old engine would retain its per-bit work, while adopting another decoder would complicate SVAC3 compatibility; neither is a stronger approach for this hot path.

Files changed (8) +757 / -298

Enhancement (4) +274 / -298
svac3_aec.cInitialize window state and account for consumed bits +22/-254

Initialize window state and account for consumed bits

• Replaces draft-shaped engine initialization with a 9-bit range and preloaded bitstream window. Retains stuffing-bit decoding and derives consumed bytes and overruns from the window marker.

source/svac3/svac3_aec.c

svac3_aec.hInline the window-based arithmetic decoder +220/-15

Inline the window-based arithmetic decoder

• Defines the refilled bitstream window, arithmetic decisions, weighted decisions, bypass decoding, and existing model updates as inline operations. Adds helpers for bit accounting and explicitly marking corrupt parser input.

source/svac3/svac3_aec.h

svac3_patch.cReduce SRCC context work and use new accounting +28/-29

Reduce SRCC context work and use new accounting

• Replaces repeated coefficient-history scans with five-bit masks and a count table, and computes position classes from per-block row and column zones. Uses the new AEC corruption and overrun helpers for patch status.

source/svac3/svac3_patch.c

svac3_recon.cSkip dequantisation arithmetic for zero coefficients +4/-0

Skip dequantisation arithmetic for zero coefficients

• Writes zero directly for zero-valued quantized coefficients, avoiding unnecessary scaling and clipping work.

source/svac3/svac3_recon.c

Tests (4) +483 / -0
CMakeLists.txtRegister differential AEC test +4/-0

Register differential AEC test

• Builds test_aec with the preserved reference decoder, links it against svac3d, and registers it with CTest.

CMakeLists.txt

aec_ref.cPreserve the draft-shaped AEC engine for comparison +299/-0

Preserve the draft-shaped AEC engine for comparison

• Adds the previous bit-by-bit arithmetic decoder and probability-model logic under reference-specific names. It provides an independent implementation for differential testing.

tests/aec_ref.c

aec_ref.hDeclare reference decoder state and API +61/-0

Declare reference decoder state and API

• Defines the draft-shaped engine and context structures and exposes its decoding functions to the differential test.

tests/aec_ref.h

test_aec.cCheck AEC equivalence across randomized streams +119/-0

Check AEC equivalence across randomized streams

• Compares bins and model state across decisions, weighted decisions, both reference bypass forms, and stuffing bits with MAEC on and off. Checks the value-range invariant and consumed-bit agreement at patch ends.

tests/test_aec.c

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Bypass bits prematurely end test streams ✓ Resolved
Description
end treats every kind >= 15 result of 1 as a stuffing bit, although that kind selects an
ordinary bypass decision during the first 31 operations. A bypass 1 in that window stops the stream
and counts as a patch end, leaving its later bins unchecked and inflating the reported patch-end
count.
Code

tests/test_aec.c[100]

+            const int end = kind >= 15 && b1 == 1;
Evidence
For kind == 15 and op <= 30, the test calls bypass decoders, but its end predicate depends only
on kind and the returned bit. The production parser uses a stuffing decision, not a bypass
decision, at an LCU boundary.

tests/test_aec.c[87-109]
source/svac3/svac3_patch.c[1390-1400]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The test treats an early bypass result of 1 as a patch-ending stuffing bit, prematurely stopping some streams.
## Fix Focus Areas
- tests/test_aec.c[87-109]
## Recommended Fix
Track whether the selected operation actually decoded a stuffing bit, and use that flag for both `end` and the patch-end count.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Weighted model errors escape the test ✓ Resolved
Description
ctx_same compares only rc[c] with fc[c], even though the weighted-decision branch also updates
context w. If the implementations update only that second context differently, the iteration can
pass when the stream ends before w is sampled again.
Code

tests/test_aec.c[R97-99]

+            const int ctx_same = rc[c].mps == fc[c].mps && rc[c].cycno == fc[c].cycno &&
+                                 rc[c].lgPmps == fc[c].lgPmps && rc[c].lgPmps0 == fc[c].lgPmps0 &&
+                                 rc[c].lgPmps1 == fc[c].lgPmps1;
Evidence
The test passes distinct c and w contexts to the weighted decoders but checks only c
afterward; the new decoder explicitly updates both contexts.

tests/test_aec.c[81-83]
tests/test_aec.c[97-104]
source/svac3/svac3_aec.h[246-249]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The equivalence test omits the second model updated by a weighted decision.
## Fix Focus Areas
- tests/test_aec.c[81-83]
- tests/test_aec.c[97-104]
## Recommended Fix
After each weighted decision, compare every field of `rc[w]` and `fc[w]` as well as the existing `c` comparison, and fail the test on either mismatch.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can add REVIEW.md to your repo root and Qodo follows it on every PR

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread tests/test_aec.c Resolved
Comment thread tests/test_aec.c Resolved
A bypass 1 early in a stream was taken for a stuffing bit and ended it,
and a two-model decision's second model went unchecked. Each operation
now compares all the models, and only a stuffing bit of 1 ends a stream:
36.7 million bins in 5000 streams, 3390 patch ends.
@widgetii

Copy link
Copy Markdown
Member Author

Thanks, both valid; both fixed in the latest commit (test only, no decoder change).

1. Bypass bits prematurely end test streams: fixed. The end-of-patch condition now depends on the stuffing branch having actually run (stuffed && b1 == 1), not on the operation kind. A bypass 1 in the first 31 operations no longer ends the stream or counts as a patch end. The rerun checks more bins in fewer, true patch ends: 36.7 million bins in 5,000 streams, 3,390 patch ends, all agreeing.

2. Weighted model errors escape the test: fixed. After every operation all 24 models are compared, so the second model of a two-model decision is checked as soon as it is updated.

Also, for the speed table: on the idle Core Ultra 5 245K, pinned to one P-core, this branch decodes faster single-threaded than the vendor decoder (svac3dec 1.0.0.27 under wine, -t 1, its own decode-loop timer, no output written):

stream this branch vendor -t 1 vendor -t 4
1080p CBR (90 pictures) 0.43 s, 209 fps 0.51 s, 178 fps 0.29 s
1080p SmartP (45) 0.21 s 0.25 s 0.16 s
1080p QP 22 ×20 (120) 1.56 s 1.83 s 1.42 s
2560x1440 (295) 3.22 s, 92 fps 3.33 s, 89 fps 2.08 s

Our times are wall-clock and include process start and file reading. The vendor's 4-thread figure is ahead because libsvac3d decodes on one thread.

@widgetii
widgetii merged commit 585f0bc into master Oct 29, 2026
5 checks passed
@widgetii
widgetii deleted the svac3-aec branch October 29, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant