Repository navigation
svac3: decoding on several threads - #11
Conversation
svac3d_set_threads() (svac3dec -t N, FFmpeg's -threads): - parsing and reconstruction run side by side. The parser emits a record per coding unit (modes, motion vector differences, QP state, the coefficients) and a reconstruction thread consumes them in order. The two keep separate maps of decoded units, since availability for prediction means reconstructed; with one thread the records are reconstructed as they are made, as before. - the loop filters run in bands on a thread pool: deblocking in 64-row bands, all vertical edges and then all horizontal ones, and SAO by rows of LCUs, which touch disjoint samples. The output is the same whatever the number of threads. A decoder keeps each finished picture's memory (maps, SAO parameters, record chunks) for the next: freeing and allocating several MB per picture cost tens of thousands of page faults. The FFmpeg wrapper passes thread_count through (0: one thread per CPU); libsvac3d is 0.2.0. ctest decodes every golden stream on 4 threads too; a new CI job runs the threaded decoder under ThreadSanitizer.
PR Summary by QodoAdd deterministic multithreaded SVAC3 decoding
AI Description
Diagram
High-Level Assessment
Files changed (17)
|
Code Review by Qodo
1.
|
- a picture rejected at begin frees what it took from the cache - pthread_cond_init() is checked; on failure the picture runs unthreaded - at most 64 threads (SVAC3_MAX_THREADS) - the reconstruction thread stops on a flag set once every record is published, not on an end record: no allocation, nothing overwritten - svac3d links Threads::Threads, and libsvac3d.pc lists -lpthread - records keep the motion vector differences in 32 bits, as parsed, so derive_mvp() clips the same sum as before
|
Thanks. All six valid; all fixed in the latest commit.
Rechecked: 34/34 in release, output identical at |
Multi-threaded decoding:
svac3d_set_threads(),svac3dec -t N, and FFmpeg's-threads. The output is bit-identical whatever the number of threads, since every stage keeps the decoding order.Parse ∥ reconstruct
parser_tand its own map of decoded units, because "available for prediction" means reconstructed. All the existing derivation, prediction and filter code runs unchanged on the reconstruction side.Loop filters on a pool (
svac3_pool.{c,h}, a small fork-join pool)Picture memory reuse: the decoder keeps each finished picture's memory (two unit maps, SAO parameters, record chunks) for the next picture (
svac3_picture_cache_t). Freeing and allocating several MB per picture had cost about 170,000 page faults on the 90-picture stream, against 5,000 now, and that was a 25% slowdown on one thread.FFmpeg: the wrapper passes
thread_countto libsvac3d (0 means one thread per CPU) and declaresAV_CODEC_CAP_OTHER_THREADS. The library version is now 0.2.0, which the patch'sconfigurerequires.Checks
recon_md5_t4_*).test_apichecks the threaded API gives the same pictures. 34/34 pass in release, in ASan+UBSan, and on AArch64.tsan: the threaded decoder on every corpus stream, plustest_api, under ThreadSanitizer. Clean locally.-threads 0and-threads 4.Speed: idle Core Ultra 5 245K, P-cores, best of 3;
masteris single-threaded:On a Cortex-X925: 0.43 / 0.32 / 0.30 s on 1080p CBR, and 1.43 / 1.02 / 0.98 s on QP 22 ×20.
What limits scaling: beyond 2 threads, parse and reconstruction are each one thread, and they take about equal time. The loop filters still run after the whole picture. The next step is to filter in a wavefront, a couple of LCU rows behind reconstruction, so the filters overlap parse and reconstruction too.