Repository navigation
svac3: pictures decoded side by side - #13
Merged
Merged
Conversation
With N threads, N pictures are decoded at a time. The caller's thread keeps the picture buffer and the reference lists in decoding order: at a picture's first patch it works out the references and a slot, and once the patches are all in it hands the picture, with a copy of its headers and patch data, to a picture thread. Slots that a picture in flight uses are held until it is output. A P picture waits only for the reference rows its motion reaches. Before each prediction it asks for the rows the 8-tap luma and 4-tap chroma interpolation read, and before co-located motion it asks for that row of 16x16 temporal motion. The patch decoder now reports how far a picture is: the rows no later filter step can change (8 above the last SAO band, 6 above the last deblocked one without SAO) and the temporal-motion rows, stored per LCU row. Each picture thread also runs its picture's reconstruction and filters behind its parser, as one thread with a pool of 3 did, which is what speeds up I pictures. Pictures come out in decoding order, with the same bits for any number of threads, through an output function, up to N pictures late. svac3d_flush() returns the rest at the end of the stream, and svac3d_decode_tag() tags each picture with the buffer its first patch came in. A picture that fails reports its error when it is output, and leaves the buffer; the pictures made from it fail too. One thread decodes as before. The FFmpeg wrapper drains at EOF (AV_CODEC_CAP_DELAY) and takes each frame's pts, dts and duration from its own packet by tag. The library version is 0.3.0. On a six-P-core Core Ultra 5 245K, at 4 threads: 1080p CBR 0.24 s -> 0.13 s, 2560x1440 1.56 s -> 0.79 s, QP 22 (an I picture in six) 0.95 s -> 0.67 s.
PR Summary by QodoDecode SVAC3 pictures concurrently with row-level reference waits
AI Description
Diagram
High-Level Assessment
Files changed (12)
|
Code Review by Qodo
1.
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
A picture handed to a picture thread is in the buffer for the pictures after it to reference, but prunes it only once it has decoded, in decoding order, and then only of the pictures taken before it. One that fails prunes nothing and leaves the buffer, as on one thread. A picture whose reference has since failed, or been decoded again, has its references found again among the pictures before it and is decoded again on the caller's thread once nothing in flight reads it; one not yet handed out has them found again first. A library picture replacing one with the same index no longer makes the pictures in flight lose it. A damaged I picture in the library stream gave 44 pictures on one thread and 40 on four; it now gives 44 on both. test_api -lost damages a P picture and the second I picture, and compares 1 and 4 threads (also on the library stream, now in ctest and the TSan job). A slot decoded on the caller's thread is marked complete, so turning threads on mid-stream with it as a reference no longer dereferences a NULL decoder; test_api changes the number at an access unit both ways. A reference read above the picture is of its first row: the wait asks for at least one row. Recycled tasks free patch buffers the next picture need not reuse. svac3dec reports a lost patch by NAL unit with threads, and the FFmpeg wrapper keeps at most 256 packets' timing.
Member
Author
|
Review addressed in a91d4f6:
Bit-exact at 2, 4 and 8 threads on every stream. TSan, ASan/UBSan and the ARM64 ctest are clean. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With N threads, N pictures are decoded at a time. The caller's thread
keeps the picture buffer and the reference lists in decoding order: at a
picture's first patch it works out the references and a slot, and once
the patches are all in it hands the picture, with a copy of its headers
and patch data, to a picture thread. Slots that a picture in flight uses
are held until it is output.
A P picture waits only for the reference rows its motion reaches. Before
each prediction it asks for the rows the 8-tap luma and 4-tap chroma
interpolation read, and before co-located motion it asks for that row of
16x16 temporal motion. The patch decoder now reports how far a picture is:
the rows no later filter step can change (8 above the last SAO band, 6
above the last deblocked one without SAO) and the temporal-motion rows,
stored per LCU row. Each picture thread also runs its picture's
reconstruction and filters behind its parser, as one thread with a pool
of 3 did, which is what speeds up I pictures.
Pictures come out in decoding order, with the same bits for any number of
threads, through an output function, up to N pictures late.
svac3d_flush() returns the rest at the end of the stream, and
svac3d_decode_tag() tags each picture with the buffer its first patch came
in. A picture that fails reports its error when it is output, and leaves
the buffer; the pictures made from it fail too. One thread decodes as
before.
The FFmpeg wrapper drains at EOF (AV_CODEC_CAP_DELAY) and takes each
frame's pts, dts and duration from its own packet by tag. The library
version is 0.3.0.
On a six-P-core Core Ultra 5 245K, at 4 threads: 1080p CBR 0.24 s -> 0.13 s,
2560x1440 1.56 s -> 0.79 s, QP 22 (an I picture in six) 0.95 s -> 0.67 s.
Verification
-threads1, 2, 4 and 8. Timestamps are identical between 1 and 4 threads.test_apiandtest_api -lost. ASan/UBSan ctest passes, plus threaded decodes.test_apicovers:svac3d_flush(), whole stream and access unit by access unit;fuzz_frames(3 picture threads). The TSan job now also runs 2 threads and the lost-patch test.Note: with N threads, up to about 3N threads can be busy, because each picture runs its own recon and filter threads. Within a fixed core budget this was as fast or faster than one thread per picture on every stream measured.